[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:54:07 dist-test-slave-nx7s krb5kdc[32001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:54:07 dist-test-slave-nx7s krb5kdc[32001](info): set up 2 sockets
Sep 25 10:54:07 dist-test-slave-nx7s krb5kdc[32001](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260925 10:54:08.437206 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.967s user 0.002s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:54:08 dist-test-slave-nx7s krb5kdc[32001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:54:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:54:08.609642 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41517
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:40065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:41517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:08.924172 32014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:08.924501 32014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:08.924636 32014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:08.932168 32014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:08.932313 32014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:08.932390 32014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:08.932463 32014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:08.932526 32014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:54:08.942718 32014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:40065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:41517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:08.944087 32014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:08.945547 32014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 10:54:08.958918 32014 server_base.cc:1034] running on GCE node
W20260925 10:54:08.958791 32020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:08.958863 32019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:08.958828 32022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:08.960083 32014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:08.961684 32014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:08.962993 32014 hybrid_clock.cc:648] HybridClock initialized: now 1790333648962930 us; error 42 us; skew 500 ppm
I20260925 10:54:08.963351 32014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:08 dist-test-slave-nx7s krb5kdc[32001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333648, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:08.966558 32014 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:54:08.968293 32014 webserver.cc:460] Webserver started at http://127.31.62.190:37581/ using document root <none> and password file <none>
I20260925 10:54:08.969022 32014 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:08.969141 32014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:08.969441 32014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:08.972483 32014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "2c5fc68dbee64b659b585276d8bdf21c"
format_stamp: "Formatted at 2026-09-25 10:54:08 on dist-test-slave-nx7s"
server_key: "3d1be7d513a06f234ac6b2f6f673fefc"
server_key_iv: "b51845d8416c1289b16cd65275dbd22b"
server_key_version: "encryptionkey@0"
I20260925 10:54:08.973318 32014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "2c5fc68dbee64b659b585276d8bdf21c"
format_stamp: "Formatted at 2026-09-25 10:54:08 on dist-test-slave-nx7s"
server_key: "3d1be7d513a06f234ac6b2f6f673fefc"
server_key_iv: "b51845d8416c1289b16cd65275dbd22b"
server_key_version: "encryptionkey@0"
I20260925 10:54:08.978652 32014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:54:08.982529 32029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:08.983984 32014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:54:08.984180 32014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "2c5fc68dbee64b659b585276d8bdf21c"
format_stamp: "Formatted at 2026-09-25 10:54:08 on dist-test-slave-nx7s"
server_key: "3d1be7d513a06f234ac6b2f6f673fefc"
server_key_iv: "b51845d8416c1289b16cd65275dbd22b"
server_key_version: "encryptionkey@0"
I20260925 10:54:08.984369 32014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:08.999694 32014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:09.004098 32014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:09.004374 32014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:09.021823 32014 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:41517
I20260925 10:54:09.021842 32080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:41517 every 8 connection(s)
I20260925 10:54:09.023232 32014 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:09.026376 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 32014
I20260925 10:54:09.026739 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:09.027889 32081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:09.029414 31994 external_mini_cluster.cc:1428] Setting key 1731cdff398a450960ec98dcdc59d4d6
I20260925 10:54:09.042384 32081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c: Bootstrap starting.
I20260925 10:54:09.046392 32081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:09.047514 32081 log.cc:824] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:09.050976 32081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c: No bootstrap required, opened a new log
I20260925 10:54:09.060524 32081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c5fc68dbee64b659b585276d8bdf21c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41517 } }
I20260925 10:54:09.061040 32081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:09.061147 32081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c5fc68dbee64b659b585276d8bdf21c, State: Initialized, Role: FOLLOWER
I20260925 10:54:09.061873 32081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c5fc68dbee64b659b585276d8bdf21c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41517 } }
I20260925 10:54:09.062144 32081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:09.062287 32081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:09.062471 32081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [term 0 FOLLOWER]: Advancing to term 1
Sep 25 10:54:09 dist-test-slave-nx7s krb5kdc[32001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:09.064962 32081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c5fc68dbee64b659b585276d8bdf21c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41517 } }
I20260925 10:54:09.065577 32081 leader_election.cc:302] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c5fc68dbee64b659b585276d8bdf21c; no voters:
I20260925 10:54:09.066316 32081 leader_election.cc:288] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:09.066553 32086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:09.067358 32086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [term 1 LEADER]: Becoming Leader. State: Replica: 2c5fc68dbee64b659b585276d8bdf21c, State: Running, Role: LEADER
I20260925 10:54:09.067924 32086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c5fc68dbee64b659b585276d8bdf21c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41517 } }
I20260925 10:54:09.070116 32087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c5fc68dbee64b659b585276d8bdf21c. Latest consensus state: current_term: 1 leader_uuid: "2c5fc68dbee64b659b585276d8bdf21c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c5fc68dbee64b659b585276d8bdf21c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41517 } } }
I20260925 10:54:09.070510 32087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:09.071213 32081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:09.076212 32090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:09.075999 32088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c5fc68dbee64b659b585276d8bdf21c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c5fc68dbee64b659b585276d8bdf21c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41517 } } }
I20260925 10:54:09.076498 32088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:09.078157 32083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:09.034107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34358 (local address 127.31.62.190:41517)
0925 10:54:09.035006 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:09.035023 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:09.036279 (+ 1256us) server_negotiation.cc:407] Connection header received
0925 10:54:09.047715 (+ 11436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:09.047740 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:09.048244 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:09.048732 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:09.052722 (+ 3990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:09.054374 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:09.056822 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:09.057605 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:09.057772 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:09.064653 (+ 6881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:09.064666 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:09.064695 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:09.064745 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:09.069123 (+ 4378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:09.070837 (+ 1714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:09.070846 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:09.070854 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:09.071069 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:09.071705 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:09.071710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:09.071713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:09.072396 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:09.072698 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:09.076020 (+ 3322us) server_negotiation.cc:299] Negotiation successful
0925 10:54:09.076509 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":370,"threads_started":1}
I20260925 10:54:09.081661 32090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:09.093118 32090 catalog_manager.cc:1347] Generated new cluster ID: e00fb1f1d93d45f8983d6588879f94ad
I20260925 10:54:09.093351 32090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:09.110011 32090 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:09.111369 32090 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:09.125124 32090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c5fc68dbee64b659b585276d8bdf21c: Generated new TSK 0
I20260925 10:54:09.126353 32090 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:54:09.173563 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41517
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:40065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:09.503767 32107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:09.504074 32107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:09.504200 32107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:09.511471 32107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:09.511605 32107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:09.511761 32107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:54:09.521523 32107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:40065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41517
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:09.522831 32107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:09.524176 32107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:09.537348 32112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:09.537407 32113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:09.537537 32107 server_base.cc:1034] running on GCE node
W20260925 10:54:09.537431 32115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:09.538640 32107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:09.539561 32107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:09.540771 32107 hybrid_clock.cc:648] HybridClock initialized: now 1790333649540720 us; error 43 us; skew 500 ppm
I20260925 10:54:09.541141 32107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:09 dist-test-slave-nx7s krb5kdc[32001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333649, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:09.544476 32107 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:54:09.546092 32107 webserver.cc:460] Webserver started at http://127.31.62.129:43029/ using document root <none> and password file <none>
I20260925 10:54:09.546795 32107 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:09.546916 32107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:09.547207 32107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:09.550223 32107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "69607f7e82584395aa786d1194dbb669"
format_stamp: "Formatted at 2026-09-25 10:54:09 on dist-test-slave-nx7s"
server_key: "6a85e42911b9a23e9e5ad26817bff22d"
server_key_iv: "38689459ff5eb2114c72f9e87fcc9e1c"
server_key_version: "encryptionkey@0"
I20260925 10:54:09.550969 32107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "69607f7e82584395aa786d1194dbb669"
format_stamp: "Formatted at 2026-09-25 10:54:09 on dist-test-slave-nx7s"
server_key: "6a85e42911b9a23e9e5ad26817bff22d"
server_key_iv: "38689459ff5eb2114c72f9e87fcc9e1c"
server_key_version: "encryptionkey@0"
I20260925 10:54:09.556272 32107 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 10:54:09.559989 32122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:09.561460 32107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:54:09.561612 32107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "69607f7e82584395aa786d1194dbb669"
format_stamp: "Formatted at 2026-09-25 10:54:09 on dist-test-slave-nx7s"
server_key: "6a85e42911b9a23e9e5ad26817bff22d"
server_key_iv: "38689459ff5eb2114c72f9e87fcc9e1c"
server_key_version: "encryptionkey@0"
I20260925 10:54:09.561808 32107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:09.583223 32107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:09.587914 32107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:09.588183 32107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:09.589210 32107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:09.590587 32107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:09.590669 32107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:09.590801 32107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:09.590852 32107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:09.620688 32107 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:37405
I20260925 10:54:09.620757 32234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:37405 every 8 connection(s)
I20260925 10:54:09.622031 32107 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:54:09.631580 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 32107
I20260925 10:54:09.631814 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:54:09.632439 31994 external_mini_cluster.cc:1428] Setting key 40afce033b938814b470f8423d95d807
Sep 25 10:54:09 dist-test-slave-nx7s krb5kdc[32001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333649, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:09.642068 32237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:09.624642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:46551 (local address 127.31.62.190:41517)
0925 10:54:09.625134 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:09.625139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:09.626324 (+ 1185us) server_negotiation.cc:407] Connection header received
0925 10:54:09.628185 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:09.628190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:09.628285 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:09.628380 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:09.630136 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:09.630858 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:09.632540 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:09.633022 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:09.633141 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:09.636558 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:09.636563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:09.636566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:09.636599 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:09.639031 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:09.639780 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:09.639784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:09.639786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:09.639904 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:09.640324 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:09.640328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:09.640330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:09.640596 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:09.640730 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:09.641471 (+ 741us) server_negotiation.cc:299] Negotiation successful
0925 10:54:09.641692 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":302,"threads_started":1}
I20260925 10:54:09.643092 32238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:09.625342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41517 (local address 127.31.62.129:46551)
0925 10:54:09.626107 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:09.626153 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:09.627880 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:09.628648 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:09.628657 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:09.629331 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:09.629947 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:09.629966 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:09.631026 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:09.631030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:09.632385 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:09.632399 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:09.633268 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:09.633276 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:09.633508 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:09.634244 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:09.634272 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:09.636337 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:09.639181 (+ 2844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:09.639190 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:09.639207 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:09.639632 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:09.640009 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:09.640014 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:09.640017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:09.640194 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:09.640712 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:09.640721 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:09.641109 (+ 388us) client_negotiation.cc:769] Sending connection context
0925 10:54:09.641369 (+ 260us) client_negotiation.cc:240] Negotiation successful
0925 10:54:09.641901 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":299,"threads_started":1}
I20260925 10:54:09.644713 32235 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41517
I20260925 10:54:09.645105 32235 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:09.646035 32235 heartbeater.cc:511] Master 127.31.62.190:41517 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
I20260925 10:54:09.648392 32046 ts_manager.cc:194] Registered new tserver with Master: 69607f7e82584395aa786d1194dbb669 (127.31.62.129:37405)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
I20260925 10:54:09.650133 32046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:46551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:54:09.671958 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41517
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:40065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:09.990705 32241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:09.990988 32241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:09.991108 32241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:09.998340 32241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:09.998451 32241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:09.998593 32241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:54:10.008487 32241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:40065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41517
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:10.009847 32241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:10.011157 32241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:10.024183 32246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:10.024214 32247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:10.024181 32249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:10.025005 32241 server_base.cc:1034] running on GCE node
I20260925 10:54:10.025621 32241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:10.026577 32241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:10.027784 32241 hybrid_clock.cc:648] HybridClock initialized: now 1790333650027723 us; error 57 us; skew 500 ppm
I20260925 10:54:10.028087 32241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:10 dist-test-slave-nx7s krb5kdc[32001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333650, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:10.031050 32241 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:54:10.032491 32241 webserver.cc:460] Webserver started at http://127.31.62.130:46233/ using document root <none> and password file <none>
I20260925 10:54:10.033190 32241 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:10.033294 32241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:10.033581 32241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:10.036422 32241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "f1d2a4688bc644418b18dc04f6208db7"
format_stamp: "Formatted at 2026-09-25 10:54:10 on dist-test-slave-nx7s"
server_key: "0f57533b8095e4d6188c8ea4830e4fc2"
server_key_iv: "84447bdd3988b0b0705074697d6a53fb"
server_key_version: "encryptionkey@0"
I20260925 10:54:10.037228 32241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "f1d2a4688bc644418b18dc04f6208db7"
format_stamp: "Formatted at 2026-09-25 10:54:10 on dist-test-slave-nx7s"
server_key: "0f57533b8095e4d6188c8ea4830e4fc2"
server_key_iv: "84447bdd3988b0b0705074697d6a53fb"
server_key_version: "encryptionkey@0"
I20260925 10:54:10.042183 32241 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:54:10.045787 32256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:10.047622 32241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:54:10.047794 32241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "f1d2a4688bc644418b18dc04f6208db7"
format_stamp: "Formatted at 2026-09-25 10:54:10 on dist-test-slave-nx7s"
server_key: "0f57533b8095e4d6188c8ea4830e4fc2"
server_key_iv: "84447bdd3988b0b0705074697d6a53fb"
server_key_version: "encryptionkey@0"
I20260925 10:54:10.047983 32241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:10.067302 32241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:10.071524 32241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:10.071779 32241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:10.072700 32241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:10.073987 32241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:10.074054 32241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:10.074141 32241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:10.074184 32241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:10.103437 32241 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:43671
I20260925 10:54:10.103482 32368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:43671 every 8 connection(s)
I20260925 10:54:10.104647 32241 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:54:10.108206 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 32241
I20260925 10:54:10.108402 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:54:10.108947 31994 external_mini_cluster.cc:1428] Setting key 257d7911aabfcefc32a6a48ea92465e8
Sep 25 10:54:10 dist-test-slave-nx7s krb5kdc[32001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333650, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
I20260925 10:54:10.126482 32237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:10.107254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:36249 (local address 127.31.62.190:41517)
0925 10:54:10.107421 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:10.107425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:10.108778 (+ 1353us) server_negotiation.cc:407] Connection header received
0925 10:54:10.110804 (+ 2026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:10.110810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:10.110885 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:10.111000 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:10.112961 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.113702 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.115732 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.116497 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.116650 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:10.120176 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:10.120184 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:10.120189 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:10.120233 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:10.123187 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:10.124067 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:10.124071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:10.124074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:10.124292 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:10.124776 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:10.124785 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:10.124787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:10.125189 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:10.125350 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:10.126035 (+ 685us) server_negotiation.cc:299] Negotiation successful
0925 10:54:10.126221 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
I20260925 10:54:10.127578 32371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:10.107666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41517 (local address 127.31.62.130:36249)
0925 10:54:10.108567 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:10.108624 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:10.110556 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:10.111237 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:10.111248 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:10.111971 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:10.112764 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:10.112790 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.113847 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.113852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:10.115567 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:10.115579 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.116741 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.116747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:10.116988 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:10.117652 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:10.117699 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:10.119911 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:10.123332 (+ 3421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:10.123342 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:10.123362 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:10.123923 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:10.124402 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:10.124407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:10.124411 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:10.124650 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:10.125332 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:10.125338 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:10.125706 (+ 368us) client_negotiation.cc:769] Sending connection context
0925 10:54:10.125949 (+ 243us) client_negotiation.cc:240] Negotiation successful
0925 10:54:10.126407 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":299,"threads_started":1}
I20260925 10:54:10.129315 32369 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41517
I20260925 10:54:10.129634 32369 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:10.130376 32369 heartbeater.cc:511] Master 127.31.62.190:41517 requested a full tablet report, sending...
I20260925 10:54:10.131837 32046 ts_manager.cc:194] Registered new tserver with Master: f1d2a4688bc644418b18dc04f6208db7 (127.31.62.130:43671)
I20260925 10:54:10.132717 32046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:36249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:54:10.147837 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41517
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:40065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:10.465489 32374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:10.465812 32374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:10.465952 32374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:10.473413 32374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:10.473549 32374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:10.473711 32374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:54:10.483721 32374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:40065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41517
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:10.485150 32374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:10.486547 32374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:10.499429 32380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:10.499459 32382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:10.499491 32379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:10.500327 32374 server_base.cc:1034] running on GCE node
I20260925 10:54:10.500991 32374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:10.501927 32374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:10.503175 32374 hybrid_clock.cc:648] HybridClock initialized: now 1790333650503105 us; error 52 us; skew 500 ppm
I20260925 10:54:10.503520 32374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:10 dist-test-slave-nx7s krb5kdc[32001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333650, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:10.506706 32374 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:54:10.508314 32374 webserver.cc:460] Webserver started at http://127.31.62.131:34537/ using document root <none> and password file <none>
I20260925 10:54:10.509059 32374 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:10.509210 32374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:10.509526 32374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:10.512426 32374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "4b4ceecf2ba2429d9922fb6550c7f724"
format_stamp: "Formatted at 2026-09-25 10:54:10 on dist-test-slave-nx7s"
server_key: "1482664c21d6890af3f690fcb9d00c24"
server_key_iv: "6ff0fb7fac26ea9ff0f368e137e2d96c"
server_key_version: "encryptionkey@0"
I20260925 10:54:10.513195 32374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "4b4ceecf2ba2429d9922fb6550c7f724"
format_stamp: "Formatted at 2026-09-25 10:54:10 on dist-test-slave-nx7s"
server_key: "1482664c21d6890af3f690fcb9d00c24"
server_key_iv: "6ff0fb7fac26ea9ff0f368e137e2d96c"
server_key_version: "encryptionkey@0"
I20260925 10:54:10.518307 32374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260925 10:54:10.521906 32389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:10.523520 32374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 10:54:10.523723 32374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "4b4ceecf2ba2429d9922fb6550c7f724"
format_stamp: "Formatted at 2026-09-25 10:54:10 on dist-test-slave-nx7s"
server_key: "1482664c21d6890af3f690fcb9d00c24"
server_key_iv: "6ff0fb7fac26ea9ff0f368e137e2d96c"
server_key_version: "encryptionkey@0"
I20260925 10:54:10.523909 32374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:10.547705 32374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:10.552052 32374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:10.552323 32374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:10.553484 32374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:10.554821 32374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:10.554899 32374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:10.555011 32374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:10.555060 32374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:10.584827 32374 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:41799
I20260925 10:54:10.584925 32501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:41799 every 8 connection(s)
I20260925 10:54:10.586280 32374 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:54:10.594082 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 32374
I20260925 10:54:10.594291 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:54:10.594806 31994 external_mini_cluster.cc:1428] Setting key 3ea84c660bfca320d9dcbad693fa260e
Sep 25 10:54:10 dist-test-slave-nx7s krb5kdc[32001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333650, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:10.607355 32237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:10.588976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:57875 (local address 127.31.62.190:41517)
0925 10:54:10.589142 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:10.589146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:10.590496 (+ 1350us) server_negotiation.cc:407] Connection header received
0925 10:54:10.592338 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:10.592344 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:10.592424 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:10.592522 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:10.594625 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.595474 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.597311 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.597904 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.598031 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:10.601526 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:10.601531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:10.601535 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:10.601568 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:10.604019 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:10.604908 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:10.604914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:10.604918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:10.605062 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:10.605541 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:10.605545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:10.605548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:10.605844 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:10.605956 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:10.606865 (+ 909us) server_negotiation.cc:299] Negotiation successful
0925 10:54:10.607079 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 10:54:10.608240 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:10.589534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41517 (local address 127.31.62.131:57875)
0925 10:54:10.590336 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:10.590375 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:10.592032 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:10.592827 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:10.592837 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:10.593460 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:10.594407 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:10.594445 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.595669 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.595675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:10.597190 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:10.597203 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.598126 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.598132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:10.598431 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:10.599115 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:10.599146 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:10.601311 (+ 2165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:10.604232 (+ 2921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:10.604241 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:10.604256 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:10.604719 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:10.605188 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:10.605191 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:10.605194 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:10.605394 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:10.605965 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:10.605972 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:10.606423 (+ 451us) client_negotiation.cc:769] Sending connection context
0925 10:54:10.606709 (+ 286us) client_negotiation.cc:240] Negotiation successful
0925 10:54:10.607113 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":281,"threads_started":1}
I20260925 10:54:10.610069 32502 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41517
I20260925 10:54:10.610496 32502 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:10.611279 32502 heartbeater.cc:511] Master 127.31.62.190:41517 requested a full tablet report, sending...
I20260925 10:54:10.612612 32046 ts_manager.cc:194] Registered new tserver with Master: 4b4ceecf2ba2429d9922fb6550c7f724 (127.31.62.131:41799)
I20260925 10:54:10.613459 32046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:57875
I20260925 10:54:10.623847 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 10:54:10 dist-test-slave-nx7s krb5kdc[32001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333650, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260925 10:54:10.653103 32235 heartbeater.cc:503] Master 127.31.62.190:41517 was elected leader, sending a full tablet report...
Sep 25 10:54:10 dist-test-slave-nx7s krb5kdc[32001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333650, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:54:10.667691 32509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:10.651002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40148 (local address 127.31.62.129:37405)
0925 10:54:10.651892 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:10.651902 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:10.651948 (+ 46us) server_negotiation.cc:407] Connection header received
0925 10:54:10.652009 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:10.652015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:10.652301 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:10.652443 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:10.654224 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.655511 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.656981 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.657557 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.657658 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:10.661812 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:10.661824 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:10.661846 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:10.661881 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:10.664744 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:10.665529 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:10.665537 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:10.665539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:10.665666 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:10.666126 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:10.666130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:10.666132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:10.666527 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:10.666832 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:10.667170 (+ 338us) server_negotiation.cc:299] Negotiation successful
0925 10:54:10.667384 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":551,"threads_started":1}
W20260925 10:54:10.669047 32150 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40148
Sep 25 10:54:10 dist-test-slave-nx7s krb5kdc[32001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 10:54:10 dist-test-slave-nx7s krb5kdc[32001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:54:10.712098 32509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:10.698195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.31.62.129:37405)
0925 10:54:10.698387 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:10.698394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:10.698801 (+ 407us) server_negotiation.cc:407] Connection header received
0925 10:54:10.699004 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:10.699008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:10.699104 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:10.699197 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:10.700523 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.701278 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.702537 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.702954 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.703074 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:10.707017 (+ 3943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:10.707022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:10.707026 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:10.707067 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:10.709655 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:10.710321 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:10.710326 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:10.710328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:10.710448 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:10.710809 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:10.710814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:10.710816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:10.711067 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:10.711287 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:10.711581 (+ 294us) server_negotiation.cc:299] Negotiation successful
0925 10:54:10.711790 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 10:54:10.713570 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 32107
I20260925 10:54:10.726446 32509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:10.714621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40160 (local address 127.31.62.129:37405)
0925 10:54:10.714844 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:10.714850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:10.715021 (+ 171us) server_negotiation.cc:407] Connection header received
0925 10:54:10.715167 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:10.715171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:10.715229 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:10.715298 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:10.716345 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.717176 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.718516 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.718940 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.719032 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:10.721466 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:10.721471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:10.721474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:10.721506 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:10.723913 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:10.724649 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:10.724653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:10.724655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:10.724799 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:10.725226 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:10.725230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:10.725233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:10.725518 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:10.725713 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:10.725948 (+ 235us) server_negotiation.cc:299] Negotiation successful
0925 10:54:10.726119 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260925 10:54:10.727174 32230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:10.811019 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 32107
I20260925 10:54:10.824361 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 32241
Sep 25 10:54:10 dist-test-slave-nx7s krb5kdc[32001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:54:10.840073 32516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:10.825160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52986 (local address 127.31.62.130:43671)
0925 10:54:10.825776 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:10.825783 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:10.825809 (+ 26us) server_negotiation.cc:407] Connection header received
0925 10:54:10.825878 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:10.825884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:10.826090 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:10.826225 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:10.827387 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.828454 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.829790 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.830421 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.830531 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:10.834546 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:10.834559 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:10.834576 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:10.834619 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:10.837453 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:10.838279 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:10.838286 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:10.838289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:10.838413 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:10.838734 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:10.838738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:10.838740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:10.839076 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:10.839354 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:10.839546 (+ 192us) server_negotiation.cc:299] Negotiation successful
0925 10:54:10.839775 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":353,"threads_started":1}
I20260925 10:54:10.841414 32364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:10.921998 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 32241
I20260925 10:54:10.940430 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 32374
Sep 25 10:54:10 dist-test-slave-nx7s krb5kdc[32001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:54:10.956566 32518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:10.941430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48794 (local address 127.31.62.131:41799)
0925 10:54:10.942026 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:10.942034 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:10.942061 (+ 27us) server_negotiation.cc:407] Connection header received
0925 10:54:10.942118 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:10.942124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:10.942356 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:10.942495 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:10.943772 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.944776 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.946156 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:10.946726 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:10.946819 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:10.950904 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:10.950918 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:10.950933 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:10.950973 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:10.953772 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:10.954562 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:10.954568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:10.954571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:10.954689 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:10.955052 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:10.955055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:10.955057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:10.955401 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:10.955625 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:10.955993 (+ 368us) server_negotiation.cc:299] Negotiation successful
0925 10:54:10.956240 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":310,"threads_started":1}
I20260925 10:54:10.957897 32497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:11.040887 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 32374
I20260925 10:54:11.055213 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 32014
I20260925 10:54:11.056393 32076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:11.158422 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 32014
2026-09-25T10:54:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3736 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:54:11 dist-test-slave-nx7s krb5kdc[32523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:54:11 dist-test-slave-nx7s krb5kdc[32523](info): set up 2 sockets
Sep 25 10:54:11 dist-test-slave-nx7s krb5kdc[32523](info): commencing operation
krb5kdc: starting...
W20260925 10:54:12.157121 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.001s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:54:12 dist-test-slave-nx7s krb5kdc[32523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:54:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:54:12.338624 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:34529
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:32781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:34529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:12.660252 32536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:12.660554 32536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:12.660667 32536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:12.668115 32536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:12.668244 32536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:12.668290 32536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:12.668340 32536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:12.668398 32536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:54:12.678719 32536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:32781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:34529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:34529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:12.680040 32536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:12.681384 32536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:12.694471 32541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:12.694519 32542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:12.695088 32544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:12.695473 32536 server_base.cc:1034] running on GCE node
I20260925 10:54:12.696314 32536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:12.697928 32536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:12.699152 32536 hybrid_clock.cc:648] HybridClock initialized: now 1790333652699087 us; error 50 us; skew 500 ppm
I20260925 10:54:12.699623 32536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:12 dist-test-slave-nx7s krb5kdc[32523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333652, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:12.706991 32536 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:54:12.708607 32536 webserver.cc:460] Webserver started at http://127.31.62.190:37577/ using document root <none> and password file <none>
I20260925 10:54:12.709349 32536 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:12.709460 32536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:12.709767 32536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:12.712805 32536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "9ceded86f19d4860afae1e939e546cf5"
format_stamp: "Formatted at 2026-09-25 10:54:12 on dist-test-slave-nx7s"
server_key: "8b0c71c3ec872dab1f365b0ed9429e1a"
server_key_iv: "ecc7ddc70e5d0205b8753ac169b1247e"
server_key_version: "encryptionkey@0"
I20260925 10:54:12.713682 32536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "9ceded86f19d4860afae1e939e546cf5"
format_stamp: "Formatted at 2026-09-25 10:54:12 on dist-test-slave-nx7s"
server_key: "8b0c71c3ec872dab1f365b0ed9429e1a"
server_key_iv: "ecc7ddc70e5d0205b8753ac169b1247e"
server_key_version: "encryptionkey@0"
I20260925 10:54:12.719303 32536 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:54:12.723296 32551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:12.724913 32536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 10:54:12.725095 32536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "9ceded86f19d4860afae1e939e546cf5"
format_stamp: "Formatted at 2026-09-25 10:54:12 on dist-test-slave-nx7s"
server_key: "8b0c71c3ec872dab1f365b0ed9429e1a"
server_key_iv: "ecc7ddc70e5d0205b8753ac169b1247e"
server_key_version: "encryptionkey@0"
I20260925 10:54:12.725294 32536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:12.740362 32536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:12.744463 32536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:12.744720 32536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:12.762502 32536 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:34529
I20260925 10:54:12.762537 32602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:34529 every 8 connection(s)
I20260925 10:54:12.763867 32536 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:12.765436 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 32536
I20260925 10:54:12.765666 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:12.766455 31994 external_mini_cluster.cc:1428] Setting key a1265be9c6ad0781351c7124f368b430
I20260925 10:54:12.768715 32603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 10:54:12 dist-test-slave-nx7s krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:12.782243 32603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5: Bootstrap starting.
I20260925 10:54:12.786373 32603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:12.787616 32603 log.cc:824] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:12.791172 32603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5: No bootstrap required, opened a new log
I20260925 10:54:12.791675 32606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:12.768513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38720 (local address 127.31.62.190:34529)
0925 10:54:12.769395 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:12.769411 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:12.769463 (+ 52us) server_negotiation.cc:407] Connection header received
0925 10:54:12.771614 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:12.771642 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:12.772198 (+ 556us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:12.772687 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:12.774139 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:12.776178 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:12.777660 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:12.778743 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:12.778948 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:12.783025 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:12.783041 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:12.783063 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:12.783117 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:12.786918 (+ 3801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:12.787674 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:12.787683 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:12.787692 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:12.787909 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:12.788409 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:12.788413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:12.788415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:12.789046 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:12.789346 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:12.789791 (+ 445us) server_negotiation.cc:299] Negotiation successful
0925 10:54:12.790275 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":425,"threads_started":1}
I20260925 10:54:12.799968 32603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ceded86f19d4860afae1e939e546cf5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34529 } }
I20260925 10:54:12.800372 32603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:12.800447 32603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ceded86f19d4860afae1e939e546cf5, State: Initialized, Role: FOLLOWER
I20260925 10:54:12.801014 32603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ceded86f19d4860afae1e939e546cf5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34529 } }
I20260925 10:54:12.801229 32603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:12.801329 32603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:12.801460 32603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:12.803561 32603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ceded86f19d4860afae1e939e546cf5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34529 } }
I20260925 10:54:12.804046 32603 leader_election.cc:302] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ceded86f19d4860afae1e939e546cf5; no voters:
I20260925 10:54:12.804589 32603 leader_election.cc:288] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:12.804764 32608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:12.805521 32608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [term 1 LEADER]: Becoming Leader. State: Replica: 9ceded86f19d4860afae1e939e546cf5, State: Running, Role: LEADER
I20260925 10:54:12.806023 32608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ceded86f19d4860afae1e939e546cf5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34529 } }
I20260925 10:54:12.806705 32603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:12.809422 32609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ceded86f19d4860afae1e939e546cf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ceded86f19d4860afae1e939e546cf5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34529 } } }
I20260925 10:54:12.809401 32610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ceded86f19d4860afae1e939e546cf5. Latest consensus state: current_term: 1 leader_uuid: "9ceded86f19d4860afae1e939e546cf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ceded86f19d4860afae1e939e546cf5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34529 } } }
I20260925 10:54:12.810030 32610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:12.809943 32609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:12.811614 32618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:12.816125 32618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:12.824548 32618 catalog_manager.cc:1347] Generated new cluster ID: 19c8594a3f5647b38c10726d0cb5e725
I20260925 10:54:12.824680 32618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:12.841080 32618 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:12.842469 32618 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:12.860430 32618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ceded86f19d4860afae1e939e546cf5: Generated new TSK 0
I20260925 10:54:12.861501 32618 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:54:12.913108 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34529
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:32781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:13.237572 32629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:13.237862 32629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:13.237957 32629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:13.245391 32629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:13.245522 32629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:13.245637 32629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:54:13.255580 32629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:32781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34529
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:13.256875 32629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:13.258301 32629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:13.271302 32635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:13.271327 32637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:13.271303 32634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:13.272531 32629 server_base.cc:1034] running on GCE node
I20260925 10:54:13.273327 32629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:13.274351 32629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:13.275574 32629 hybrid_clock.cc:648] HybridClock initialized: now 1790333653275497 us; error 68 us; skew 500 ppm
I20260925 10:54:13.275906 32629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:13 dist-test-slave-nx7s krb5kdc[32523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333653, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:13.279173 32629 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:54:13.280687 32629 webserver.cc:460] Webserver started at http://127.31.62.129:41609/ using document root <none> and password file <none>
I20260925 10:54:13.281435 32629 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:13.281555 32629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:13.281857 32629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:13.284890 32629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "e41a29dc750c414cb5ce0842c731406f"
format_stamp: "Formatted at 2026-09-25 10:54:13 on dist-test-slave-nx7s"
server_key: "872d60c0e8477a9be7dd28cc9fda5e1f"
server_key_iv: "943f9516a87b3358d3d3e28ea8b55147"
server_key_version: "encryptionkey@0"
I20260925 10:54:13.285743 32629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "e41a29dc750c414cb5ce0842c731406f"
format_stamp: "Formatted at 2026-09-25 10:54:13 on dist-test-slave-nx7s"
server_key: "872d60c0e8477a9be7dd28cc9fda5e1f"
server_key_iv: "943f9516a87b3358d3d3e28ea8b55147"
server_key_version: "encryptionkey@0"
I20260925 10:54:13.291308 32629 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260925 10:54:13.295320 32644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:13.296993 32629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260925 10:54:13.297169 32629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "e41a29dc750c414cb5ce0842c731406f"
format_stamp: "Formatted at 2026-09-25 10:54:13 on dist-test-slave-nx7s"
server_key: "872d60c0e8477a9be7dd28cc9fda5e1f"
server_key_iv: "943f9516a87b3358d3d3e28ea8b55147"
server_key_version: "encryptionkey@0"
I20260925 10:54:13.297362 32629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:13.309773 32629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:13.314090 32629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:13.314361 32629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:13.315382 32629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:13.316727 32629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:13.316805 32629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:13.316942 32629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:13.316998 32629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:13.347095 32629 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:35677
I20260925 10:54:13.347200 32756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:35677 every 8 connection(s)
I20260925 10:54:13.348445 32629 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:54:13.349571 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 32629
I20260925 10:54:13.349782 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:54:13.350328 31994 external_mini_cluster.cc:1428] Setting key ad074aeac26d50b1cdf702e6b5f07435
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
Sep 25 10:54:13 dist-test-slave-nx7s krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333653, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
I20260925 10:54:13.374022 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:13.351130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:47597 (local address 127.31.62.190:34529)
0925 10:54:13.351532 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:13.351536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:13.352779 (+ 1243us) server_negotiation.cc:407] Connection header received
0925 10:54:13.358524 (+ 5745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:13.358562 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:13.358666 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:13.358789 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:13.361506 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:13.362498 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:13.364272 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:13.364675 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:13.364793 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:13.368300 (+ 3507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:13.368307 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:13.368312 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:13.368348 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:13.370838 (+ 2490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:13.371617 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:13.371622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:13.371624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:13.371763 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:13.372207 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:13.372211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:13.372214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:13.372481 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:13.372624 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:13.373426 (+ 802us) server_negotiation.cc:299] Negotiation successful
0925 10:54:13.373656 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":195,"threads_started":1}
I20260925 10:54:13.375010 32760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:13.351755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34529 (local address 127.31.62.129:47597)
0925 10:54:13.352605 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:13.352668 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:13.357984 (+ 5316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:13.359295 (+ 1311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:13.359311 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:13.360180 (+ 869us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:13.361272 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:13.361303 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:13.362656 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:13.362662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:13.364101 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:13.364115 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:13.364947 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:13.364956 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:13.365196 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:13.365838 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:13.365862 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:13.368037 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:13.371000 (+ 2963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:13.371011 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:13.371031 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:13.371481 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:13.371868 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:13.371873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:13.371877 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:13.372091 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:13.372607 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:13.372615 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:13.373042 (+ 427us) client_negotiation.cc:769] Sending connection context
0925 10:54:13.373291 (+ 249us) client_negotiation.cc:240] Negotiation successful
0925 10:54:13.373852 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":307,"threads_started":1}
I20260925 10:54:13.376657 32757 heartbeater.cc:347] Connected to a master server at 127.31.62.190:34529
I20260925 10:54:13.376999 32757 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:13.377766 32757 heartbeater.cc:511] Master 127.31.62.190:34529 requested a full tablet report, sending...
I20260925 10:54:13.379964 32568 ts_manager.cc:194] Registered new tserver with Master: e41a29dc750c414cb5ce0842c731406f (127.31.62.129:35677)
I20260925 10:54:13.381522 32568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:47597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:54:13.393349 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34529
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:32781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:13.709404 32763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:13.709714 32763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:13.709824 32763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:13.717077 32763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:13.717195 32763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:13.717309 32763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:54:13.727052 32763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:32781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34529
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:13.728395 32763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:13.729789 32763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:13.743001 300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:13.743073 303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:13.743083 301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:13.743824 32763 server_base.cc:1034] running on GCE node
I20260925 10:54:13.744431 32763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:13.745462 32763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:13.746691 32763 hybrid_clock.cc:648] HybridClock initialized: now 1790333653746613 us; error 57 us; skew 500 ppm
I20260925 10:54:13.747011 32763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:13 dist-test-slave-nx7s krb5kdc[32523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333653, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:13.750376 32763 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:54:13.752043 32763 webserver.cc:460] Webserver started at http://127.31.62.130:46515/ using document root <none> and password file <none>
I20260925 10:54:13.752800 32763 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:13.752986 32763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:13.753361 32763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:13.756425 32763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "90de37c3edd948378b9d5a2ca4a7b1f2"
format_stamp: "Formatted at 2026-09-25 10:54:13 on dist-test-slave-nx7s"
server_key: "7b30a13d505252a1ceefeb3455519621"
server_key_iv: "744589426d5538f742e3841923912074"
server_key_version: "encryptionkey@0"
I20260925 10:54:13.757333 32763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "90de37c3edd948378b9d5a2ca4a7b1f2"
format_stamp: "Formatted at 2026-09-25 10:54:13 on dist-test-slave-nx7s"
server_key: "7b30a13d505252a1ceefeb3455519621"
server_key_iv: "744589426d5538f742e3841923912074"
server_key_version: "encryptionkey@0"
I20260925 10:54:13.763453 32763 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260925 10:54:13.768411 310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:13.770351 32763 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 10:54:13.770550 32763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "90de37c3edd948378b9d5a2ca4a7b1f2"
format_stamp: "Formatted at 2026-09-25 10:54:13 on dist-test-slave-nx7s"
server_key: "7b30a13d505252a1ceefeb3455519621"
server_key_iv: "744589426d5538f742e3841923912074"
server_key_version: "encryptionkey@0"
I20260925 10:54:13.770800 32763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:13.808151 32763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:13.812561 32763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:13.812870 32763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:13.813834 32763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:13.815146 32763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:13.815231 32763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:13.815361 32763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:13.815416 32763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:13.843920 32763 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:41391
I20260925 10:54:13.843971 422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:41391 every 8 connection(s)
I20260925 10:54:13.845400 32763 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:54:13.850438 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 32763
I20260925 10:54:13.850638 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:54:13.851166 31994 external_mini_cluster.cc:1428] Setting key 511a8b177a78788be4c5c11e7f7bbc0b
Sep 25 10:54:13 dist-test-slave-nx7s krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333653, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
I20260925 10:54:13.868135 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:13.848070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:56657 (local address 127.31.62.190:34529)
0925 10:54:13.848319 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:13.848324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:13.849559 (+ 1235us) server_negotiation.cc:407] Connection header received
0925 10:54:13.851970 (+ 2411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:13.851976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:13.852070 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:13.852168 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:13.854241 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:13.854963 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:13.857265 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:13.857748 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:13.857910 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:13.861944 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:13.861950 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:13.861953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:13.861990 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:13.864499 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:13.865612 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:13.865616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:13.865618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:13.865758 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:13.866300 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:13.866303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:13.866306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:13.866602 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:13.866768 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:13.867554 (+ 786us) server_negotiation.cc:299] Negotiation successful
0925 10:54:13.867802 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260925 10:54:13.869176 425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:13.848570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34529 (local address 127.31.62.130:56657)
0925 10:54:13.849390 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:13.849430 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:13.851619 (+ 2189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:13.852507 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:13.852517 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:13.853145 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:13.854029 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:13.854058 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:13.855122 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:13.855127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:13.857053 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:13.857069 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:13.857987 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:13.857992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:13.858205 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:13.858942 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:13.858971 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:13.861632 (+ 2661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:13.864693 (+ 3061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:13.864706 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:13.864730 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:13.865460 (+ 730us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:13.865874 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:13.865879 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:13.865882 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:13.866174 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:13.866735 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:13.866744 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:13.867119 (+ 375us) client_negotiation.cc:769] Sending connection context
0925 10:54:13.867436 (+ 317us) client_negotiation.cc:240] Negotiation successful
0925 10:54:13.868009 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":336,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
I20260925 10:54:13.870891 423 heartbeater.cc:347] Connected to a master server at 127.31.62.190:34529
I20260925 10:54:13.871174 423 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:13.871906 423 heartbeater.cc:511] Master 127.31.62.190:34529 requested a full tablet report, sending...
I20260925 10:54:13.873485 32568 ts_manager.cc:194] Registered new tserver with Master: 90de37c3edd948378b9d5a2ca4a7b1f2 (127.31.62.130:41391)
I20260925 10:54:13.874491 32568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:56657
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:54:13.891851 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34529
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:32781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:14.207386 428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:14.207677 428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:14.207795 428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:14.215248 428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:14.215366 428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:14.215483 428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:54:14.225229 428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:32781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34529
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:14.226513 428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:14.227835 428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:14.240932 433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:14.241014 436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:14.241040 434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:14.242085 428 server_base.cc:1034] running on GCE node
I20260925 10:54:14.242780 428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:14.243744 428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:14.244952 428 hybrid_clock.cc:648] HybridClock initialized: now 1790333654244884 us; error 55 us; skew 500 ppm
I20260925 10:54:14.245280 428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:14 dist-test-slave-nx7s krb5kdc[32523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333654, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:14.248430 428 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:54:14.250252 428 webserver.cc:460] Webserver started at http://127.31.62.131:33969/ using document root <none> and password file <none>
I20260925 10:54:14.250950 428 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:14.251070 428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:14.251381 428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:14.254393 428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "9ea14f2e29b84664a8b26f1dabf1da6a"
format_stamp: "Formatted at 2026-09-25 10:54:14 on dist-test-slave-nx7s"
server_key: "9dc72d87435f6dfdf0ffc7ff83d11c63"
server_key_iv: "93d1f37552ed0bbb9dbac4d42569ca60"
server_key_version: "encryptionkey@0"
I20260925 10:54:14.255151 428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "9ea14f2e29b84664a8b26f1dabf1da6a"
format_stamp: "Formatted at 2026-09-25 10:54:14 on dist-test-slave-nx7s"
server_key: "9dc72d87435f6dfdf0ffc7ff83d11c63"
server_key_iv: "93d1f37552ed0bbb9dbac4d42569ca60"
server_key_version: "encryptionkey@0"
I20260925 10:54:14.260613 428 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260925 10:54:14.264314 443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:14.265874 428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:54:14.266042 428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "9ea14f2e29b84664a8b26f1dabf1da6a"
format_stamp: "Formatted at 2026-09-25 10:54:14 on dist-test-slave-nx7s"
server_key: "9dc72d87435f6dfdf0ffc7ff83d11c63"
server_key_iv: "93d1f37552ed0bbb9dbac4d42569ca60"
server_key_version: "encryptionkey@0"
I20260925 10:54:14.266233 428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:14.292133 428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:14.296607 428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:14.296933 428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:14.297982 428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:14.299355 428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:14.299432 428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:14.299546 428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:14.299594 428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:14.329563 428 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:42659
I20260925 10:54:14.329627 555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:42659 every 8 connection(s)
I20260925 10:54:14.330928 428 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:54:14.338114 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 428
I20260925 10:54:14.338290 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:54:14.338707 31994 external_mini_cluster.cc:1428] Setting key b7ed07ad697547d7dad5edd5a9fb3649
Sep 25 10:54:14 dist-test-slave-nx7s krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333654, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:14.351392 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.333559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:46687 (local address 127.31.62.190:34529)
0925 10:54:14.333764 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:14.333768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:14.334917 (+ 1149us) server_negotiation.cc:407] Connection header received
0925 10:54:14.336805 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:14.336810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:14.336939 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:14.337027 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:14.338811 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.339636 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.341258 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.341670 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.341778 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.345449 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:14.345455 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:14.345458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:14.345491 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:14.348248 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.349098 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.349102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.349104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.349235 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.349655 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.349659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.349661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.349936 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:14.350072 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:14.350821 (+ 749us) server_negotiation.cc:299] Negotiation successful
0925 10:54:14.351049 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 10:54:14.352312 558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.334017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34529 (local address 127.31.62.131:46687)
0925 10:54:14.334737 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:14.334783 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:14.336511 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:14.337295 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:14.337305 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:14.337960 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:14.338637 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:14.338657 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.339782 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.339786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:14.341113 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:14.341132 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.341900 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.341907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:14.342155 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.342826 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:14.342850 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:14.345162 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:14.348416 (+ 3254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:14.348427 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:14.348447 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:14.348970 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:14.349343 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:14.349349 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:14.349353 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:14.349538 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:14.350070 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:14.350078 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:14.350445 (+ 367us) client_negotiation.cc:769] Sending connection context
0925 10:54:14.350679 (+ 234us) client_negotiation.cc:240] Negotiation successful
0925 10:54:14.351212 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":272,"threads_started":1}
I20260925 10:54:14.353767 556 heartbeater.cc:347] Connected to a master server at 127.31.62.190:34529
I20260925 10:54:14.354033 556 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:14.354732 556 heartbeater.cc:511] Master 127.31.62.190:34529 requested a full tablet report, sending...
I20260925 10:54:14.356063 32568 ts_manager.cc:194] Registered new tserver with Master: 9ea14f2e29b84664a8b26f1dabf1da6a (127.31.62.131:42659)
I20260925 10:54:14.356917 32568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:46687
I20260925 10:54:14.366864 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:54:14.384601 32757 heartbeater.cc:503] Master 127.31.62.190:34529 was elected leader, sending a full tablet report...
I20260925 10:54:14.386301 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.373313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38730 (local address 127.31.62.190:34529)
0925 10:54:14.373465 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:14.373471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:14.373984 (+ 513us) server_negotiation.cc:407] Connection header received
0925 10:54:14.374154 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:14.374158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:14.374226 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:14.374309 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:14.375859 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.376627 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.378010 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.378423 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.378531 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.380921 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:14.380929 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:14.380933 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:14.380973 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:14.383498 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.384219 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.384224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.384227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.384360 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.384767 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.384771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.384774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.385149 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:14.385390 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:14.385751 (+ 361us) server_negotiation.cc:299] Negotiation successful
0925 10:54:14.385980 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 10:54:14.393136 32568 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:54:14.399878 32568 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:54:14.441448 572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.412704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:42659 (local address 127.0.0.1:60846)
0925 10:54:14.430868 (+ 18164us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:14.430893 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:14.431045 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:14.433866 (+ 2821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:14.433874 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:14.433908 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:14.434409 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:14.434427 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.436503 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.436512 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:14.438470 (+ 1958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:14.438483 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.440386 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.440393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:14.440589 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.440645 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 10:54:14.440775 (+ 130us) client_negotiation.cc:240] Negotiation successful
0925 10:54:14.441014 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18003,"mutex_wait_us":156,"thread_start_us":20817,"threads_started":1}
I20260925 10:54:14.442003 569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.414730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:41391 (local address 127.0.0.1:53140)
0925 10:54:14.433866 (+ 19136us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:14.433885 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:14.434010 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:14.434835 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:14.434839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:14.434862 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:14.435167 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:14.435179 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.436977 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.436983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:14.439317 (+ 2334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:14.439330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.441262 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.441266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:14.441382 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.441407 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 10:54:14.441488 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 10:54:14.441621 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18985,"mutex_wait_us":42,"thread_start_us":15819,"threads_started":1}
I20260925 10:54:14.442028 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.414832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53140 (local address 127.31.62.130:41391)
0925 10:54:14.433034 (+ 18202us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:14.433044 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:14.433997 (+ 953us) server_negotiation.cc:407] Connection header received
0925 10:54:14.434213 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:14.434221 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:14.434443 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:14.434611 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:14.435338 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.436824 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.439484 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.440798 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.440946 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.441127 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:14.441560 (+ 433us) server_negotiation.cc:299] Negotiation successful
0925 10:54:14.441724 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17989,"thread_start_us":17889,"threads_started":1}
I20260925 10:54:14.442127 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.413072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.31.62.129:35677)
0925 10:54:14.433553 (+ 20481us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:14.433562 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:14.433591 (+ 29us) server_negotiation.cc:407] Connection header received
0925 10:54:14.433683 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:14.433689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:14.433917 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:14.434106 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:14.434796 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.436246 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.439186 (+ 2940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.440609 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.440740 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.441612 (+ 872us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:14.441682 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 10:54:14.441840 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20316,"thread_start_us":20151,"threads_started":1}
I20260925 10:54:14.442586 568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.412712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:35677 (local address 127.0.0.1:51332)
0925 10:54:14.427728 (+ 15016us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:14.427772 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:14.427988 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:14.434099 (+ 6111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:14.434104 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:14.434129 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:14.434574 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:14.434587 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.436396 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.436401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:14.438994 (+ 2593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:14.439008 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.440987 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.440993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:14.441153 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.441185 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 10:54:14.441279 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 10:54:14.441412 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14736,"mutex_wait_us":74,"thread_start_us":14570,"threads_started":1}
I20260925 10:54:14.442884 570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.412793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60846 (local address 127.31.62.131:42659)
0925 10:54:14.432633 (+ 19840us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:14.432644 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:14.432688 (+ 44us) server_negotiation.cc:407] Connection header received
0925 10:54:14.432799 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:14.432807 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:14.433296 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:14.433531 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:14.434657 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.436356 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.438628 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.439882 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.440015 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.440248 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:14.442397 (+ 2149us) server_negotiation.cc:299] Negotiation successful
0925 10:54:14.442570 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19634,"thread_start_us":20054,"threads_started":1}
I20260925 10:54:14.446066 358 tablet_service.cc:1467] Processing CreateTablet for tablet 90ccdb3a02c846f79d1d7770fa4ee6f8 (DEFAULT_TABLE table=test-table [id=99a1beba0a9a40e68cdb4373da1c9f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:14.446201 491 tablet_service.cc:1467] Processing CreateTablet for tablet 90ccdb3a02c846f79d1d7770fa4ee6f8 (DEFAULT_TABLE table=test-table [id=99a1beba0a9a40e68cdb4373da1c9f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:14.446677 32692 tablet_service.cc:1467] Processing CreateTablet for tablet 90ccdb3a02c846f79d1d7770fa4ee6f8 (DEFAULT_TABLE table=test-table [id=99a1beba0a9a40e68cdb4373da1c9f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:14.447363 358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90ccdb3a02c846f79d1d7770fa4ee6f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:14.447543 491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90ccdb3a02c846f79d1d7770fa4ee6f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:14.447906 32692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90ccdb3a02c846f79d1d7770fa4ee6f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:14.458173 574 tablet_bootstrap.cc:492] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 90de37c3edd948378b9d5a2ca4a7b1f2: Bootstrap starting.
I20260925 10:54:14.461668 575 tablet_bootstrap.cc:492] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P e41a29dc750c414cb5ce0842c731406f: Bootstrap starting.
I20260925 10:54:14.462029 574 tablet_bootstrap.cc:654] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 90de37c3edd948378b9d5a2ca4a7b1f2: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:14.463469 574 log.cc:824] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 90de37c3edd948378b9d5a2ca4a7b1f2: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:14.465337 575 tablet_bootstrap.cc:654] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P e41a29dc750c414cb5ce0842c731406f: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:14.466602 575 log.cc:824] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P e41a29dc750c414cb5ce0842c731406f: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:14.467106 574 tablet_bootstrap.cc:492] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 90de37c3edd948378b9d5a2ca4a7b1f2: No bootstrap required, opened a new log
I20260925 10:54:14.467439 574 ts_tablet_manager.cc:1397] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 90de37c3edd948378b9d5a2ca4a7b1f2: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260925 10:54:14.467689 576 tablet_bootstrap.cc:492] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a: Bootstrap starting.
I20260925 10:54:14.470211 575 tablet_bootstrap.cc:492] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P e41a29dc750c414cb5ce0842c731406f: No bootstrap required, opened a new log
I20260925 10:54:14.470494 575 ts_tablet_manager.cc:1397] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P e41a29dc750c414cb5ce0842c731406f: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260925 10:54:14.472031 576 tablet_bootstrap.cc:654] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:14.473359 576 log.cc:824] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:14.476642 576 tablet_bootstrap.cc:492] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a: No bootstrap required, opened a new log
I20260925 10:54:14.476991 576 ts_tablet_manager.cc:1397] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.002s
I20260925 10:54:14.477499 574 raft_consensus.cc:348] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 90de37c3edd948378b9d5a2ca4a7b1f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea14f2e29b84664a8b26f1dabf1da6a" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42659 } } peers { permanent_uuid: "e41a29dc750c414cb5ce0842c731406f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35677 } } peers { permanent_uuid: "90de37c3edd948378b9d5a2ca4a7b1f2" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41391 } }
I20260925 10:54:14.477980 574 raft_consensus.cc:374] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 90de37c3edd948378b9d5a2ca4a7b1f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:14.478112 574 raft_consensus.cc:729] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 90de37c3edd948378b9d5a2ca4a7b1f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de37c3edd948378b9d5a2ca4a7b1f2, State: Initialized, Role: FOLLOWER
I20260925 10:54:14.478248 575 raft_consensus.cc:348] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P e41a29dc750c414cb5ce0842c731406f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea14f2e29b84664a8b26f1dabf1da6a" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42659 } } peers { permanent_uuid: "e41a29dc750c414cb5ce0842c731406f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35677 } } peers { permanent_uuid: "90de37c3edd948378b9d5a2ca4a7b1f2" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41391 } }
I20260925 10:54:14.478642 575 raft_consensus.cc:374] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P e41a29dc750c414cb5ce0842c731406f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:14.478753 575 raft_consensus.cc:729] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P e41a29dc750c414cb5ce0842c731406f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e41a29dc750c414cb5ce0842c731406f, State: Initialized, Role: FOLLOWER
I20260925 10:54:14.478778 574 consensus_queue.cc:260] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 90de37c3edd948378b9d5a2ca4a7b1f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea14f2e29b84664a8b26f1dabf1da6a" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42659 } } peers { permanent_uuid: "e41a29dc750c414cb5ce0842c731406f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35677 } } peers { permanent_uuid: "90de37c3edd948378b9d5a2ca4a7b1f2" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41391 } }
I20260925 10:54:14.479293 575 consensus_queue.cc:260] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P e41a29dc750c414cb5ce0842c731406f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea14f2e29b84664a8b26f1dabf1da6a" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42659 } } peers { permanent_uuid: "e41a29dc750c414cb5ce0842c731406f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35677 } } peers { permanent_uuid: "90de37c3edd948378b9d5a2ca4a7b1f2" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41391 } }
I20260925 10:54:14.481026 423 heartbeater.cc:503] Master 127.31.62.190:34529 was elected leader, sending a full tablet report...
I20260925 10:54:14.481711 574 ts_tablet_manager.cc:1428] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 90de37c3edd948378b9d5a2ca4a7b1f2: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260925 10:54:14.486650 575 ts_tablet_manager.cc:1428] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P e41a29dc750c414cb5ce0842c731406f: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260925 10:54:14.488154 576 raft_consensus.cc:348] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea14f2e29b84664a8b26f1dabf1da6a" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42659 } } peers { permanent_uuid: "e41a29dc750c414cb5ce0842c731406f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35677 } } peers { permanent_uuid: "90de37c3edd948378b9d5a2ca4a7b1f2" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41391 } }
I20260925 10:54:14.488556 576 raft_consensus.cc:374] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:14.488653 576 raft_consensus.cc:729] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea14f2e29b84664a8b26f1dabf1da6a, State: Initialized, Role: FOLLOWER
I20260925 10:54:14.489321 576 consensus_queue.cc:260] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea14f2e29b84664a8b26f1dabf1da6a" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42659 } } peers { permanent_uuid: "e41a29dc750c414cb5ce0842c731406f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35677 } } peers { permanent_uuid: "90de37c3edd948378b9d5a2ca4a7b1f2" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41391 } }
I20260925 10:54:14.490377 556 heartbeater.cc:503] Master 127.31.62.190:34529 was elected leader, sending a full tablet report...
I20260925 10:54:14.490947 576 ts_tablet_manager.cc:1428] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a: Time spent starting tablet: real 0.014s user 0.011s sys 0.001s
I20260925 10:54:14.518612 582 raft_consensus.cc:482] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:54:14.518937 582 raft_consensus.cc:504] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea14f2e29b84664a8b26f1dabf1da6a" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42659 } } peers { permanent_uuid: "e41a29dc750c414cb5ce0842c731406f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35677 } } peers { permanent_uuid: "90de37c3edd948378b9d5a2ca4a7b1f2" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41391 } }
I20260925 10:54:14.520641 582 leader_election.cc:288] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e41a29dc750c414cb5ce0842c731406f (127.31.62.129:35677), 90de37c3edd948378b9d5a2ca4a7b1f2 (127.31.62.130:41391)
I20260925 10:54:14.527092 558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.520944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:41391 (local address 127.31.62.131:34379)
0925 10:54:14.521138 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:14.521158 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:14.521308 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:14.521638 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:14.521646 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:14.521679 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:14.521851 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:14.521859 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.523211 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.523217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:14.524759 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:14.524771 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.526331 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.526337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:14.526536 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.526572 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 10:54:14.526670 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 10:54:14.526813 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 10:54:14.527148 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.521087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:34379 (local address 127.31.62.130:41391)
0925 10:54:14.521324 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:14.521329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:14.521351 (+ 22us) server_negotiation.cc:407] Connection header received
0925 10:54:14.521422 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:14.521427 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:14.521501 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:14.521590 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:14.522003 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.522973 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.524940 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.526136 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.526264 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.526321 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:14.526738 (+ 417us) server_negotiation.cc:299] Negotiation successful
0925 10:54:14.526851 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260925 10:54:14.528059 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.521255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:58181 (local address 127.31.62.129:35677)
0925 10:54:14.521431 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:14.521436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:14.521943 (+ 507us) server_negotiation.cc:407] Connection header received
0925 10:54:14.522108 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:14.522112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:14.522169 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:14.522234 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:14.522865 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.523865 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.526092 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.527251 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.527405 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.527473 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:14.527805 (+ 332us) server_negotiation.cc:299] Negotiation successful
0925 10:54:14.527896 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 10:54:14.528272 583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.521110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:35677 (local address 127.31.62.131:58181)
0925 10:54:14.521845 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:14.521859 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:14.521962 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:14.522341 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:14.522346 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:14.522371 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:14.522698 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:14.522708 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.524042 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.524046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:14.525907 (+ 1861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:14.525922 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.527477 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.527483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:14.527640 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.527667 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 10:54:14.527785 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 10:54:14.527917 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":409,"threads_started":1}
I20260925 10:54:14.528330 378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90ccdb3a02c846f79d1d7770fa4ee6f8" candidate_uuid: "9ea14f2e29b84664a8b26f1dabf1da6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de37c3edd948378b9d5a2ca4a7b1f2" is_pre_election: true
I20260925 10:54:14.529033 378 raft_consensus.cc:2454] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 90de37c3edd948378b9d5a2ca4a7b1f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ea14f2e29b84664a8b26f1dabf1da6a in term 0.
I20260925 10:54:14.529249 32712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90ccdb3a02c846f79d1d7770fa4ee6f8" candidate_uuid: "9ea14f2e29b84664a8b26f1dabf1da6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e41a29dc750c414cb5ce0842c731406f" is_pre_election: true
I20260925 10:54:14.529732 32712 raft_consensus.cc:2454] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P e41a29dc750c414cb5ce0842c731406f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ea14f2e29b84664a8b26f1dabf1da6a in term 0.
I20260925 10:54:14.529784 444 leader_election.cc:302] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90de37c3edd948378b9d5a2ca4a7b1f2, 9ea14f2e29b84664a8b26f1dabf1da6a; no voters:
I20260925 10:54:14.530264 582 raft_consensus.cc:2781] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:54:14.530416 582 raft_consensus.cc:482] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:54:14.530516 582 raft_consensus.cc:3037] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:14.532658 582 raft_consensus.cc:504] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea14f2e29b84664a8b26f1dabf1da6a" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42659 } } peers { permanent_uuid: "e41a29dc750c414cb5ce0842c731406f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35677 } } peers { permanent_uuid: "90de37c3edd948378b9d5a2ca4a7b1f2" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41391 } }
I20260925 10:54:14.533285 582 leader_election.cc:288] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [CANDIDATE]: Term 1 election: Requested vote from peers e41a29dc750c414cb5ce0842c731406f (127.31.62.129:35677), 90de37c3edd948378b9d5a2ca4a7b1f2 (127.31.62.130:41391)
I20260925 10:54:14.533694 32712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90ccdb3a02c846f79d1d7770fa4ee6f8" candidate_uuid: "9ea14f2e29b84664a8b26f1dabf1da6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e41a29dc750c414cb5ce0842c731406f"
I20260925 10:54:14.533723 378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90ccdb3a02c846f79d1d7770fa4ee6f8" candidate_uuid: "9ea14f2e29b84664a8b26f1dabf1da6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de37c3edd948378b9d5a2ca4a7b1f2"
I20260925 10:54:14.533874 32712 raft_consensus.cc:3037] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P e41a29dc750c414cb5ce0842c731406f [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:14.533914 378 raft_consensus.cc:3037] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 90de37c3edd948378b9d5a2ca4a7b1f2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:14.536021 32712 raft_consensus.cc:2454] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P e41a29dc750c414cb5ce0842c731406f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ea14f2e29b84664a8b26f1dabf1da6a in term 1.
I20260925 10:54:14.536049 378 raft_consensus.cc:2454] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 90de37c3edd948378b9d5a2ca4a7b1f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ea14f2e29b84664a8b26f1dabf1da6a in term 1.
I20260925 10:54:14.536473 444 leader_election.cc:302] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90de37c3edd948378b9d5a2ca4a7b1f2, 9ea14f2e29b84664a8b26f1dabf1da6a; no voters:
I20260925 10:54:14.536808 582 raft_consensus.cc:2781] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:14.537340 582 raft_consensus.cc:686] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [term 1 LEADER]: Becoming Leader. State: Replica: 9ea14f2e29b84664a8b26f1dabf1da6a, State: Running, Role: LEADER
I20260925 10:54:14.537803 582 consensus_queue.cc:237] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea14f2e29b84664a8b26f1dabf1da6a" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42659 } } peers { permanent_uuid: "e41a29dc750c414cb5ce0842c731406f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35677 } } peers { permanent_uuid: "90de37c3edd948378b9d5a2ca4a7b1f2" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41391 } }
I20260925 10:54:14.542287 32568 catalog_manager.cc:5495] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ea14f2e29b84664a8b26f1dabf1da6a (127.31.62.131). New cstate: current_term: 1 leader_uuid: "9ea14f2e29b84664a8b26f1dabf1da6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea14f2e29b84664a8b26f1dabf1da6a" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e41a29dc750c414cb5ce0842c731406f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90de37c3edd948378b9d5a2ca4a7b1f2" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41391 } health_report { overall_health: UNKNOWN } } }
Sep 25 10:54:14 dist-test-slave-nx7s krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:54:14.573050 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.557119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51348 (local address 127.31.62.129:35677)
0925 10:54:14.557330 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:14.557335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:14.557914 (+ 579us) server_negotiation.cc:407] Connection header received
0925 10:54:14.558036 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:14.558041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:14.558124 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:14.558232 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:14.559834 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.560681 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.562121 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.562548 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.562649 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.567016 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:14.567033 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:14.567049 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:14.567093 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:14.570236 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.571076 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.571085 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.571087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.571223 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.571607 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.571610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.571613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.571987 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:14.572208 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:14.572511 (+ 303us) server_negotiation.cc:299] Negotiation successful
0925 10:54:14.572672 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260925 10:54:14.582466 557 tablet.cc:2369] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 9ea14f2e29b84664a8b26f1dabf1da6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 10:54:14 dist-test-slave-nx7s krb5kdc[32523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333654, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260925 10:54:14.597416 424 tablet.cc:2369] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P 90de37c3edd948378b9d5a2ca4a7b1f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:54:14.600744 32758 tablet.cc:2369] T 90ccdb3a02c846f79d1d7770fa4ee6f8 P e41a29dc750c414cb5ce0842c731406f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 10:54:14 dist-test-slave-nx7s krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333654, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:54:14.617805 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.603785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51354 (local address 127.31.62.129:35677)
0925 10:54:14.603999 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:14.604004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:14.604510 (+ 506us) server_negotiation.cc:407] Connection header received
0925 10:54:14.604639 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:14.604643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:14.604709 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:14.604796 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:14.606184 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.607003 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.608361 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.608761 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.608888 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.612754 (+ 3866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:14.612759 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:14.612763 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:14.612803 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:14.615265 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.615951 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.615956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.615958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.616094 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.616584 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.616588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.616590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.616929 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:14.617133 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:14.617387 (+ 254us) server_negotiation.cc:299] Negotiation successful
0925 10:54:14.617559 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Sep 25 10:54:14 dist-test-slave-nx7s krb5kdc[32523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 10:54:14 dist-test-slave-nx7s krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:54:14.661515 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.647561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51370 (local address 127.31.62.129:35677)
0925 10:54:14.647737 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:14.647743 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:14.648091 (+ 348us) server_negotiation.cc:407] Connection header received
0925 10:54:14.648262 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:14.648266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:14.648340 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:14.648424 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:14.649874 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.650549 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.651878 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.652332 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.652450 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.656624 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:14.656630 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:14.656633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:14.656668 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:14.659078 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.659691 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.659696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.659698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.659829 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.660174 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.660177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.660179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.660462 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:14.660676 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:14.660979 (+ 303us) server_negotiation.cc:299] Negotiation successful
0925 10:54:14.661195 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260925 10:54:14.663245 32672 tablet_service.cc:2908] Rejecting scan request for tablet 90ccdb3a02c846f79d1d7770fa4ee6f8: Uninitialized: clean time has not yet been initialized
I20260925 10:54:14.666448 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 32629
I20260925 10:54:14.679052 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.667407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51376 (local address 127.31.62.129:35677)
0925 10:54:14.667561 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:14.667565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:14.667863 (+ 298us) server_negotiation.cc:407] Connection header received
0925 10:54:14.668010 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:14.668015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:14.668125 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:14.668233 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:14.669425 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.670190 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.671450 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.671854 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.671968 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.674139 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:14.674144 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:14.674147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:14.674178 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:14.676554 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.677244 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.677248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.677251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.677368 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.677705 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.677709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.677711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.677999 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:14.678251 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:14.678543 (+ 292us) server_negotiation.cc:299] Negotiation successful
0925 10:54:14.678777 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 10:54:14.679697 32752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:14.767580 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 32629
I20260925 10:54:14.786434 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 32763
Sep 25 10:54:14 dist-test-slave-nx7s krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:54:14.801038 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.787190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53154 (local address 127.31.62.130:41391)
0925 10:54:14.787392 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:14.787398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:14.787417 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:54:14.787578 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:14.787591 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:14.787693 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:14.787794 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:14.789066 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.789820 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.791060 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.791447 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.791559 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.795457 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:14.795474 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:14.795490 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:14.795528 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:14.798306 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.798997 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.799005 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.799008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.799134 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.799501 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.799505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.799507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.799906 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:14.800208 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:14.800470 (+ 262us) server_negotiation.cc:299] Negotiation successful
0925 10:54:14.800687 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260925 10:54:14.801717 418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:14.886853 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 32763
I20260925 10:54:14.900507 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 428
Sep 25 10:54:14 dist-test-slave-nx7s krb5kdc[32523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:54:14.915206 570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:14.901329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60860 (local address 127.31.62.131:42659)
0925 10:54:14.901502 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:14.901507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:14.901524 (+ 17us) server_negotiation.cc:407] Connection header received
0925 10:54:14.901612 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:14.901616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:14.901674 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:14.901762 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:14.903047 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.903769 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.905164 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:14.905567 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:14.905660 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:14.909662 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:14.909677 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:14.909694 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:14.909742 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:14.912514 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.913215 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.913222 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.913224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.913359 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:14.913723 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:14.913726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:14.913728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:14.914128 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:14.914400 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:14.914680 (+ 280us) server_negotiation.cc:299] Negotiation successful
0925 10:54:14.914905 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 10:54:14.915856 551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:15.003970 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 428
I20260925 10:54:15.016901 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 32536
I20260925 10:54:15.018074 32598 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:15.125221 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 32536
2026-09-25T10:54:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (3967 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:54:15 dist-test-slave-nx7s krb5kdc[603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:54:15 dist-test-slave-nx7s krb5kdc[603](info): set up 2 sockets
Sep 25 10:54:15 dist-test-slave-nx7s krb5kdc[603](info): commencing operation
krb5kdc: starting...
W20260925 10:54:16.129052 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:54:16 dist-test-slave-nx7s krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:54:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:54:16.302582 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:40663
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:40663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:16.619112 616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:16.619441 616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:16.619563 616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:16.627012 616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:16.627154 616 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 10:54:16.627221 616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:16.627305 616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:16.627367 616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:16.627452 616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:54:16.637488 616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:40663
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:40663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:16.638774 616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:16.640074 616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:16.651386 622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:16.651315 624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:16.651350 621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:16.651999 616 server_base.cc:1034] running on GCE node
I20260925 10:54:16.653009 616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:16.654448 616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:16.655648 616 hybrid_clock.cc:648] HybridClock initialized: now 1790333656655514 us; error 108 us; skew 500 ppm
I20260925 10:54:16.655969 616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:16 dist-test-slave-nx7s krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333656, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:16.659053 616 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:54:16.660481 616 webserver.cc:460] Webserver started at http://127.31.62.190:42861/ using document root <none> and password file <none>
I20260925 10:54:16.661238 616 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:16.661334 616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:16.661638 616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:16.664485 616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "8967a1520d804688a34036c4526d9b79"
format_stamp: "Formatted at 2026-09-25 10:54:16 on dist-test-slave-nx7s"
server_key: "7a5717bbebf7cc6c37f089374f74e3ef"
server_key_iv: "41f9798c2e143f7c2dec84a94cf3a7b0"
server_key_version: "encryptionkey@0"
I20260925 10:54:16.665262 616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "8967a1520d804688a34036c4526d9b79"
format_stamp: "Formatted at 2026-09-25 10:54:16 on dist-test-slave-nx7s"
server_key: "7a5717bbebf7cc6c37f089374f74e3ef"
server_key_iv: "41f9798c2e143f7c2dec84a94cf3a7b0"
server_key_version: "encryptionkey@0"
I20260925 10:54:16.670814 616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:54:16.674489 631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:16.675994 616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:54:16.676172 616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "8967a1520d804688a34036c4526d9b79"
format_stamp: "Formatted at 2026-09-25 10:54:16 on dist-test-slave-nx7s"
server_key: "7a5717bbebf7cc6c37f089374f74e3ef"
server_key_iv: "41f9798c2e143f7c2dec84a94cf3a7b0"
server_key_version: "encryptionkey@0"
I20260925 10:54:16.676470 616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:16.694278 616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:16.698781 616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:16.699048 616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:16.719226 616 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:40663
I20260925 10:54:16.719305 692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:40663 every 8 connection(s)
I20260925 10:54:16.720650 616 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:16.724822 693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:16.729209 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 616
I20260925 10:54:16.729440 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:16.730232 31994 external_mini_cluster.cc:1428] Setting key 507d3d91c1dde6461ddaa31d655ec9c5
I20260925 10:54:16.737190 693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79: Bootstrap starting.
I20260925 10:54:16.741034 693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:16.742224 693 log.cc:824] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:16.745028 693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79: No bootstrap required, opened a new log
Sep 25 10:54:16 dist-test-slave-nx7s krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:16.753453 693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8967a1520d804688a34036c4526d9b79" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40663 } }
I20260925 10:54:16.753868 693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:16.753963 693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8967a1520d804688a34036c4526d9b79, State: Initialized, Role: FOLLOWER
I20260925 10:54:16.754674 696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:16.732452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53350 (local address 127.31.62.190:40663)
0925 10:54:16.733281 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:16.733294 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:16.733340 (+ 46us) server_negotiation.cc:407] Connection header received
0925 10:54:16.735518 (+ 2178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:16.735547 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:16.736024 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:16.736425 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:16.738312 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:16.739763 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:16.741553 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:16.742292 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:16.742474 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:16.746336 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:16.746344 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:16.746361 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:16.746400 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:16.749889 (+ 3489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:16.750725 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:16.750737 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:16.750744 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:16.750952 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:16.751383 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:16.751386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:16.751391 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:16.752050 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:16.752361 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:16.752904 (+ 543us) server_negotiation.cc:299] Negotiation successful
0925 10:54:16.753372 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":399,"threads_started":1}
I20260925 10:54:16.754654 693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8967a1520d804688a34036c4526d9b79" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40663 } }
I20260925 10:54:16.754947 693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:16.755074 693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:16.755249 693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:16.757668 693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8967a1520d804688a34036c4526d9b79" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40663 } }
I20260925 10:54:16.758247 693 leader_election.cc:302] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8967a1520d804688a34036c4526d9b79; no voters:
I20260925 10:54:16.758806 693 leader_election.cc:288] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:16.758965 698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:16.759871 698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [term 1 LEADER]: Becoming Leader. State: Replica: 8967a1520d804688a34036c4526d9b79, State: Running, Role: LEADER
I20260925 10:54:16.760540 698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8967a1520d804688a34036c4526d9b79" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40663 } }
I20260925 10:54:16.761773 693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:16.763495 700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8967a1520d804688a34036c4526d9b79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8967a1520d804688a34036c4526d9b79" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40663 } } }
I20260925 10:54:16.763924 700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:16.763892 699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8967a1520d804688a34036c4526d9b79. Latest consensus state: current_term: 1 leader_uuid: "8967a1520d804688a34036c4526d9b79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8967a1520d804688a34036c4526d9b79" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40663 } } }
I20260925 10:54:16.766582 699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:16.770802 703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:16.777837 703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:16.788743 703 catalog_manager.cc:1347] Generated new cluster ID: 2fe8f5c9fda44644934c288477f4a8a6
I20260925 10:54:16.788921 703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:16.811205 703 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:16.813208 703 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:16.825525 703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79: Generated new TSK 0
I20260925 10:54:16.826772 703 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:54:16.874444 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:40663
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:17.200068 719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:17.200353 719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:17.200476 719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:17.207816 719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:17.207943 719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:17.208052 719 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 10:54:17.208150 719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:54:17.218101 719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.62.190:40663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:17.219409 719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:17.220752 719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:17.234603 725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:17.234624 724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:17.234627 727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:17.235575 719 server_base.cc:1034] running on GCE node
I20260925 10:54:17.236310 719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:17.237504 719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:17.238785 719 hybrid_clock.cc:648] HybridClock initialized: now 1790333657238692 us; error 45 us; skew 500 ppm
I20260925 10:54:17.239112 719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:17 dist-test-slave-nx7s krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333657, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:17.242496 719 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:54:17.244520 719 webserver.cc:460] Webserver started at http://127.31.62.129:41853/ using document root <none> and password file <none>
I20260925 10:54:17.245379 719 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:17.245507 719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:17.245854 719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:17.249137 719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "387792af2d024535994dd2bc9b75c447"
format_stamp: "Formatted at 2026-09-25 10:54:17 on dist-test-slave-nx7s"
server_key: "76cb6f30d6d9a9f2bf6ac73dacf3aa36"
server_key_iv: "0c121125eec2328a766e0cd1625cce77"
server_key_version: "encryptionkey@0"
I20260925 10:54:17.250082 719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "387792af2d024535994dd2bc9b75c447"
format_stamp: "Formatted at 2026-09-25 10:54:17 on dist-test-slave-nx7s"
server_key: "76cb6f30d6d9a9f2bf6ac73dacf3aa36"
server_key_iv: "0c121125eec2328a766e0cd1625cce77"
server_key_version: "encryptionkey@0"
I20260925 10:54:17.256515 719 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260925 10:54:17.260572 734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:17.262380 719 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:54:17.262568 719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "387792af2d024535994dd2bc9b75c447"
format_stamp: "Formatted at 2026-09-25 10:54:17 on dist-test-slave-nx7s"
server_key: "76cb6f30d6d9a9f2bf6ac73dacf3aa36"
server_key_iv: "0c121125eec2328a766e0cd1625cce77"
server_key_version: "encryptionkey@0"
I20260925 10:54:17.262833 719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:17.297740 719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:17.302040 719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:17.302263 719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:17.304795 719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:17.304920 719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:17.305056 719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:17.305114 719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 10:54:17 dist-test-slave-nx7s krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333657, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:17.328799 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:17.305550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:37701 (local address 127.31.62.190:40663)
0925 10:54:17.305987 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:17.305993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:17.307298 (+ 1305us) server_negotiation.cc:407] Connection header received
0925 10:54:17.310132 (+ 2834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:17.310139 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:17.310250 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:17.310379 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:17.312932 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.313877 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.316077 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.316625 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.316746 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:17.321808 (+ 5062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:17.321813 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:17.321816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:17.321849 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:17.324713 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:17.325838 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:17.325843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:17.325846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:17.325971 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:17.326537 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:17.326541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:17.326543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:17.326858 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:17.326983 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:17.328019 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0925 10:54:17.328279 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":210,"threads_started":1}
I20260925 10:54:17.330444 744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:17.306232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40663 (local address 127.31.62.129:37701)
0925 10:54:17.307073 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:17.307137 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:17.309767 (+ 2630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:17.310694 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:17.310706 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:17.311650 (+ 944us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:17.312691 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:17.312721 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.314039 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.314045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:17.315915 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:17.315933 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.316961 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.316972 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:17.317334 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:17.318212 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:17.318249 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:17.321508 (+ 3259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:17.324898 (+ 3390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:17.324914 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:17.324937 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:17.325704 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:17.326074 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:17.326080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:17.326084 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:17.326387 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:17.326979 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:17.326989 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:17.327532 (+ 543us) client_negotiation.cc:769] Sending connection context
0925 10:54:17.327865 (+ 333us) client_negotiation.cc:240] Negotiation successful
0925 10:54:17.328500 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":307,"threads_started":1}
I20260925 10:54:17.348603 719 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:44159
I20260925 10:54:17.348686 854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:44159 every 8 connection(s)
I20260925 10:54:17.350148 719 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:54:17.352159 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 719
I20260925 10:54:17.352375 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:54:17.352696 855 heartbeater.cc:347] Connected to a master server at 127.31.62.190:40663
I20260925 10:54:17.353093 855 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:17.353173 31994 external_mini_cluster.cc:1428] Setting key 5ce1451afcf383d89540ed1786d9801c
I20260925 10:54:17.353389 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:17.337307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:49625 (local address 127.31.62.190:40663)
0925 10:54:17.337478 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:17.337486 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:17.338255 (+ 769us) server_negotiation.cc:407] Connection header received
0925 10:54:17.338469 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:17.338475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:17.338567 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:17.338686 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:17.340027 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.340920 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.343176 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.343775 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.343926 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:17.346679 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:17.346685 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:17.346689 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:17.346726 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:17.349529 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:17.350471 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:17.350477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:17.350481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:17.350663 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:17.351220 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:17.351225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:17.351228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:17.351607 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:17.351825 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:17.352818 (+ 993us) server_negotiation.cc:299] Negotiation successful
0925 10:54:17.353092 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 10:54:17.353960 855 heartbeater.cc:511] Master 127.31.62.190:40663 requested a full tablet report, sending...
I20260925 10:54:17.363893 648 ts_manager.cc:194] Registered new tserver with Master: 387792af2d024535994dd2bc9b75c447 (127.31.62.129:44159)
I20260925 10:54:17.365851 648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:37701
I20260925 10:54:17.356961 825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:17.337399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40663 (local address 127.31.62.129:49625)
0925 10:54:17.338112 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:17.338135 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:17.338322 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:17.338869 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:17.338876 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:17.339514 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:17.339850 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:17.339862 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.341066 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.341072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:17.342941 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:17.342960 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.344001 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.344010 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:17.344226 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:17.344965 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:17.344993 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:17.346448 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:17.349701 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:17.349707 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:17.349710 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:17.350272 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:17.350842 (+ 570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:17.350849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:17.350853 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:17.351046 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:17.351881 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:17.351884 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:17.352086 (+ 202us) client_negotiation.cc:769] Sending connection context
0925 10:54:17.352228 (+ 142us) client_negotiation.cc:240] Negotiation successful
0925 10:54:17.352599 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":384,"threads_started":1}
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:54:17.404068 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:40663
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:17.723584 859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:17.723867 859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:17.723987 859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:17.731380 859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:17.731496 859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:17.731603 859 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 10:54:17.731709 859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:54:17.741824 859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.62.190:40663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:17.743052 859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:17.744307 859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:17.757236 867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:17.757405 864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:17.757375 865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:17.757964 859 server_base.cc:1034] running on GCE node
I20260925 10:54:17.758634 859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:17.759615 859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:17.760864 859 hybrid_clock.cc:648] HybridClock initialized: now 1790333657760765 us; error 61 us; skew 500 ppm
I20260925 10:54:17.761181 859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:17 dist-test-slave-nx7s krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333657, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:17.769570 859 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:54:17.771173 859 webserver.cc:460] Webserver started at http://127.31.62.130:43951/ using document root <none> and password file <none>
I20260925 10:54:17.771912 859 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:17.772020 859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:17.772327 859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:17.775352 859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "00ec3d8f228746c3b3c8116be6aaeca3"
format_stamp: "Formatted at 2026-09-25 10:54:17 on dist-test-slave-nx7s"
server_key: "cbb1f6de75f92e1b1fcfc1d6d8e4a9e4"
server_key_iv: "e519f0f13ed2b519ad6ae33724645f9d"
server_key_version: "encryptionkey@0"
I20260925 10:54:17.776182 859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "00ec3d8f228746c3b3c8116be6aaeca3"
format_stamp: "Formatted at 2026-09-25 10:54:17 on dist-test-slave-nx7s"
server_key: "cbb1f6de75f92e1b1fcfc1d6d8e4a9e4"
server_key_iv: "e519f0f13ed2b519ad6ae33724645f9d"
server_key_version: "encryptionkey@0"
I20260925 10:54:17.781809 859 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260925 10:54:17.786005 874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:17.787534 859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 10:54:17.787724 859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "00ec3d8f228746c3b3c8116be6aaeca3"
format_stamp: "Formatted at 2026-09-25 10:54:17 on dist-test-slave-nx7s"
server_key: "cbb1f6de75f92e1b1fcfc1d6d8e4a9e4"
server_key_iv: "e519f0f13ed2b519ad6ae33724645f9d"
server_key_version: "encryptionkey@0"
I20260925 10:54:17.787954 859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:17.803126 859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:17.807526 859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:17.807799 859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:17.810683 859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:17.810794 859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:17.810925 859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:17.810992 859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 10:54:17 dist-test-slave-nx7s krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333657, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:17.834791 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:17.811407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:37787 (local address 127.31.62.190:40663)
0925 10:54:17.811666 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:17.811672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:17.813349 (+ 1677us) server_negotiation.cc:407] Connection header received
0925 10:54:17.815529 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:17.815535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:17.815619 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:17.815708 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:17.818065 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.819078 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.822015 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.822655 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.822773 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:17.827459 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:17.827466 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:17.827470 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:17.827517 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:17.830535 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:17.831721 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:17.831728 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:17.831731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:17.831876 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:17.832460 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:17.832465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:17.832469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:17.832878 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:17.833030 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:17.834074 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0925 10:54:17.834327 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260925 10:54:17.835836 883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:17.812106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40663 (local address 127.31.62.130:37787)
0925 10:54:17.813138 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:17.813194 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:17.815177 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:17.816001 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:17.816016 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:17.816843 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:17.817781 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:17.817818 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.819284 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.819293 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:17.821796 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:17.821827 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.823120 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.823129 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:17.823343 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:17.824116 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:17.824142 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:17.827128 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:17.830759 (+ 3631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:17.830785 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:17.830810 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:17.831554 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:17.831991 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:17.831996 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:17.832000 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:17.832338 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:17.833023 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:17.833035 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:17.833562 (+ 527us) client_negotiation.cc:769] Sending connection context
0925 10:54:17.833938 (+ 376us) client_negotiation.cc:240] Negotiation successful
0925 10:54:17.834498 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":456,"threads_started":1}
I20260925 10:54:17.856703 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:17.841785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:48937 (local address 127.31.62.190:40663)
0925 10:54:17.841972 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:17.841978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:17.842700 (+ 722us) server_negotiation.cc:407] Connection header received
0925 10:54:17.842875 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:17.842879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:17.842944 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:17.843025 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:17.844316 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.845137 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.847290 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.848014 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.848244 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:17.850317 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:17.850324 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:17.850328 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:17.850370 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:17.853632 (+ 3262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:17.854296 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:17.854302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:17.854306 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:17.854471 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:17.854952 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:17.854958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:17.854961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:17.855438 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:17.855604 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:17.856136 (+ 532us) server_negotiation.cc:299] Negotiation successful
0925 10:54:17.856369 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 10:54:17.856890 956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:17.841885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40663 (local address 127.31.62.130:48937)
0925 10:54:17.842568 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:17.842590 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:17.842756 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:17.843068 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:17.843073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:17.843674 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:17.844075 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:17.844089 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.845378 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.845385 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:17.847118 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:17.847135 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:17.848279 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:17.848287 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:17.848451 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:17.849107 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:17.849127 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:17.850091 (+ 964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:17.853760 (+ 3669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:17.853765 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:17.853770 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:17.854174 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:17.854627 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:17.854632 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:17.854636 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:17.854806 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:17.855574 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:17.855579 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:17.855835 (+ 256us) client_negotiation.cc:769] Sending connection context
0925 10:54:17.856014 (+ 179us) client_negotiation.cc:240] Negotiation successful
0925 10:54:17.856404 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":351,"threads_started":1}
I20260925 10:54:17.858829 859 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:46163
I20260925 10:54:17.858855 993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:46163 every 8 connection(s)
I20260925 10:54:17.860088 859 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:54:17.861259 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 859
I20260925 10:54:17.861449 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:54:17.861946 31994 external_mini_cluster.cc:1428] Setting key e19bdcf45fd3043135e5ebfcf2ce83ce
I20260925 10:54:17.862269 994 heartbeater.cc:347] Connected to a master server at 127.31.62.190:40663
I20260925 10:54:17.862509 994 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:17.863375 994 heartbeater.cc:511] Master 127.31.62.190:40663 requested a full tablet report, sending...
I20260925 10:54:17.864878 648 ts_manager.cc:194] Registered new tserver with Master: 00ec3d8f228746c3b3c8116be6aaeca3 (127.31.62.130:46163)
I20260925 10:54:17.865669 648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:37787
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:54:17.901332 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:40663
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:18.221093 998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:18.221414 998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:18.221529 998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:18.229337 998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:18.229470 998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:18.229579 998 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 10:54:18.229691 998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:54:18.239945 998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.62.190:40663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:18.241338 998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:18.242726 998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:18.256374 1003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:18.256460 1004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:18.256626 1006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:18.256460 998 server_base.cc:1034] running on GCE node
I20260925 10:54:18.257618 998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:18.258651 998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:18.259889 998 hybrid_clock.cc:648] HybridClock initialized: now 1790333658259812 us; error 48 us; skew 500 ppm
I20260925 10:54:18.260205 998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:18 dist-test-slave-nx7s krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333658, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:18.263530 998 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:54:18.265079 998 webserver.cc:460] Webserver started at http://127.31.62.131:39127/ using document root <none> and password file <none>
I20260925 10:54:18.265826 998 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:18.265934 998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:18.266242 998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:18.269316 998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "b58aaebaa14049a393c051bf16e5a221"
format_stamp: "Formatted at 2026-09-25 10:54:18 on dist-test-slave-nx7s"
server_key: "868f00b19dd4d1a9155127131e9a9d0d"
server_key_iv: "59756fc7a3735b49142b95eb1813e493"
server_key_version: "encryptionkey@0"
I20260925 10:54:18.270087 998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "b58aaebaa14049a393c051bf16e5a221"
format_stamp: "Formatted at 2026-09-25 10:54:18 on dist-test-slave-nx7s"
server_key: "868f00b19dd4d1a9155127131e9a9d0d"
server_key_iv: "59756fc7a3735b49142b95eb1813e493"
server_key_version: "encryptionkey@0"
I20260925 10:54:18.275449 998 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:54:18.279381 1013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:18.281188 998 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260925 10:54:18.281378 998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "b58aaebaa14049a393c051bf16e5a221"
format_stamp: "Formatted at 2026-09-25 10:54:18 on dist-test-slave-nx7s"
server_key: "868f00b19dd4d1a9155127131e9a9d0d"
server_key_iv: "59756fc7a3735b49142b95eb1813e493"
server_key_version: "encryptionkey@0"
I20260925 10:54:18.281634 998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:18.307370 998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:18.312137 998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:18.312459 998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:18.315397 998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:18.315521 998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:18.315672 998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:18.315750 998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 10:54:18 dist-test-slave-nx7s krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333658, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:18.343202 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.316117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:53729 (local address 127.31.62.190:40663)
0925 10:54:18.316314 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:18.316320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:18.318138 (+ 1818us) server_negotiation.cc:407] Connection header received
0925 10:54:18.320121 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:18.320126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:18.320216 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:18.320324 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:18.323036 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.323980 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.327352 (+ 3372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.327979 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.328157 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.334057 (+ 5900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:18.334063 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:18.334066 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:18.334105 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:18.338383 (+ 4278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:18.339695 (+ 1312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:18.339700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:18.339702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:18.339867 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:18.340636 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:18.340642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:18.340645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:18.341061 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:18.341246 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:18.342543 (+ 1297us) server_negotiation.cc:299] Negotiation successful
0925 10:54:18.342832 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 10:54:18.345110 1022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.317084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40663 (local address 127.31.62.131:53729)
0925 10:54:18.317965 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:18.318004 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:18.319825 (+ 1821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:18.320685 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:18.320700 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:18.321652 (+ 952us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:18.322742 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.322775 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.324221 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.324231 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.327083 (+ 2852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.327113 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.328340 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.328354 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.328788 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.330027 (+ 1239us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:18.330080 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:18.333741 (+ 3661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:18.338642 (+ 4901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:18.338706 (+ 64us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:18.338735 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:18.339478 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:18.340021 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:18.340028 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:18.340035 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:18.340453 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:18.341231 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:18.341242 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:18.341935 (+ 693us) client_negotiation.cc:769] Sending connection context
0925 10:54:18.342379 (+ 444us) client_negotiation.cc:240] Negotiation successful
0925 10:54:18.343133 (+ 754us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":405,"threads_started":1}
I20260925 10:54:18.360458 998 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:36413
I20260925 10:54:18.360522 1132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:36413 every 8 connection(s)
I20260925 10:54:18.362169 998 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:54:18.364389 1133 heartbeater.cc:347] Connected to a master server at 127.31.62.190:40663
I20260925 10:54:18.364696 1133 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:18.365656 1133 heartbeater.cc:511] Master 127.31.62.190:40663 requested a full tablet report, sending...
I20260925 10:54:18.367360 648 ts_manager.cc:194] Registered new tserver with Master: b58aaebaa14049a393c051bf16e5a221 (127.31.62.131:36413)
I20260925 10:54:18.368232 648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:53729
I20260925 10:54:18.368651 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 998
I20260925 10:54:18.368837 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:54:18.369338 31994 external_mini_cluster.cc:1428] Setting key aca52a9bb7fefb833f7b0d3934b0b727
I20260925 10:54:18.370138 855 heartbeater.cc:503] Master 127.31.62.190:40663 was elected leader, sending a full tablet report...
I20260925 10:54:18.370434 1121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.355391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40663 (local address 127.31.62.131:35089)
0925 10:54:18.356063 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:18.356082 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:18.356248 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:18.356720 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:18.356725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:18.357436 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:18.357932 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.357951 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.359401 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.359406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.360998 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.361012 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.361715 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.361725 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.361901 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.362489 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:18.362515 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:18.363842 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:18.367487 (+ 3645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:18.367492 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:18.367496 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:18.367930 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:18.368327 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:18.368333 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:18.368338 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:18.368497 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:18.369211 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:18.369215 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:18.369457 (+ 242us) client_negotiation.cc:769] Sending connection context
0925 10:54:18.369615 (+ 158us) client_negotiation.cc:240] Negotiation successful
0925 10:54:18.369979 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":578,"threads_started":1}
I20260925 10:54:18.371338 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.355787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:35089 (local address 127.31.62.190:40663)
0925 10:54:18.355973 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:18.355979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:18.356196 (+ 217us) server_negotiation.cc:407] Connection header received
0925 10:54:18.356392 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:18.356397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:18.356501 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:18.356649 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:18.358129 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.359200 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.361144 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.361533 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.361681 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.364054 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:18.364060 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:18.364063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:18.364108 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:18.367329 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:18.368048 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:18.368054 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:18.368059 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:18.368210 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:18.368605 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:18.368611 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:18.368614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:18.369052 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:18.369222 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:18.369733 (+ 511us) server_negotiation.cc:299] Negotiation successful
0925 10:54:18.370821 (+ 1088us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 10:54:18.372586 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 10:54:18 dist-test-slave-nx7s krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 10:54:18 dist-test-slave-nx7s krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:18.419283 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.404616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53352 (local address 127.31.62.190:40663)
0925 10:54:18.404907 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:18.404913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:18.405042 (+ 129us) server_negotiation.cc:407] Connection header received
0925 10:54:18.405309 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:18.405313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:18.405378 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:18.405465 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:18.406828 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.407569 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.408838 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.409315 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.409433 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.413542 (+ 4109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:18.413548 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:18.413552 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:18.413604 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:18.416299 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:18.417129 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:18.417136 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:18.417138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:18.417289 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:18.417720 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:18.417723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:18.417726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:18.418032 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:18.418301 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:18.418629 (+ 328us) server_negotiation.cc:299] Negotiation successful
0925 10:54:18.418864 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260925 10:54:18.423826 647 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:54:18.430927 647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:54:18.468299 1147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.443903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51914 (local address 127.31.62.130:46163)
0925 10:54:18.459712 (+ 15809us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:18.459722 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:18.459757 (+ 35us) server_negotiation.cc:407] Connection header received
0925 10:54:18.459869 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:18.459877 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:18.460186 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:18.460415 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:18.461401 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.462986 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.465484 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.466949 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.467088 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.467311 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:18.467771 (+ 460us) server_negotiation.cc:299] Negotiation successful
0925 10:54:18.467943 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15665,"thread_start_us":15552,"threads_started":1}
I20260925 10:54:18.468408 1149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.443719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:46163 (local address 127.0.0.1:51914)
0925 10:54:18.456378 (+ 12659us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:18.456439 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:18.456678 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:18.460502 (+ 3824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:18.460512 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:18.460557 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:18.461156 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.461168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.463150 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.463156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.465306 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.465321 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.467182 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.467190 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.467468 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.467531 (+ 63us) client_negotiation.cc:769] Sending connection context
0925 10:54:18.467666 (+ 135us) client_negotiation.cc:240] Negotiation successful
0925 10:54:18.467925 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12326,"mutex_wait_us":136,"thread_start_us":14590,"threads_started":1}
I20260925 10:54:18.468684 1146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.444104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:44159 (local address 127.0.0.1:35112)
0925 10:54:18.458013 (+ 13909us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:18.458039 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:18.458209 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:18.460693 (+ 2484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:18.460698 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:18.460730 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:18.461273 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.461286 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.463412 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.463418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.465450 (+ 2032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.465464 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.467381 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.467388 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.467573 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.467607 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 10:54:18.467665 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 10:54:18.467915 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13731,"mutex_wait_us":13,"thread_start_us":16671,"threads_started":1}
I20260925 10:54:18.469699 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.444629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35112 (local address 127.31.62.129:44159)
0925 10:54:18.460063 (+ 15434us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:18.460071 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:18.460098 (+ 27us) server_negotiation.cc:407] Connection header received
0925 10:54:18.460186 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:18.460194 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:18.460435 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:18.461487 (+ 1052us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:18.461762 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.463264 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.465597 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.467118 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.468931 (+ 1813us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.469164 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:18.469248 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 10:54:18.469415 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15293,"thread_start_us":15186,"threads_started":1}
I20260925 10:54:18.470753 1148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.444336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:36413 (local address 127.0.0.1:36170)
0925 10:54:18.461116 (+ 16780us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:18.461142 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:18.461296 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:18.462256 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:18.462260 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:18.462285 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:18.462769 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.462782 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.464811 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.464886 (+ 75us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.467295 (+ 2409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.467312 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.469185 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.469192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.469377 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.469415 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 10:54:18.469511 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 10:54:18.469638 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16637,"thread_start_us":11545,"threads_started":1}
I20260925 10:54:18.472518 924 tablet_service.cc:1467] Processing CreateTablet for tablet 753da43281cb4e058e147774f6bfd202 (DEFAULT_TABLE table=test-table [id=fae6daa801884b788aef4f1c13fac292]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:18.473459 1150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.444581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36170 (local address 127.31.62.131:36413)
0925 10:54:18.458904 (+ 14323us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:18.458915 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:18.461289 (+ 2374us) server_negotiation.cc:407] Connection header received
0925 10:54:18.461502 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:18.461508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:18.461757 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:18.461956 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:18.462994 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.464607 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.467478 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.468985 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.469109 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.469316 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:18.469575 (+ 259us) server_negotiation.cc:299] Negotiation successful
0925 10:54:18.473045 (+ 3470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14105,"thread_start_us":14341,"threads_started":1}
I20260925 10:54:18.473094 785 tablet_service.cc:1467] Processing CreateTablet for tablet 753da43281cb4e058e147774f6bfd202 (DEFAULT_TABLE table=test-table [id=fae6daa801884b788aef4f1c13fac292]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:18.474120 924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 753da43281cb4e058e147774f6bfd202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:18.474637 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 753da43281cb4e058e147774f6bfd202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:18.476653 1063 tablet_service.cc:1467] Processing CreateTablet for tablet 753da43281cb4e058e147774f6bfd202 (DEFAULT_TABLE table=test-table [id=fae6daa801884b788aef4f1c13fac292]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:18.477932 1063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 753da43281cb4e058e147774f6bfd202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:18.488900 1152 tablet_bootstrap.cc:492] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3: Bootstrap starting.
I20260925 10:54:18.491410 1153 tablet_bootstrap.cc:492] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447: Bootstrap starting.
I20260925 10:54:18.493095 1154 tablet_bootstrap.cc:492] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221: Bootstrap starting.
I20260925 10:54:18.493947 1152 tablet_bootstrap.cc:654] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:18.495005 1152 log.cc:824] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:18.495959 1154 tablet_bootstrap.cc:654] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:18.497231 1154 log.cc:824] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:18.498152 1152 tablet_bootstrap.cc:492] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3: No bootstrap required, opened a new log
I20260925 10:54:18.498413 1152 ts_tablet_manager.cc:1397] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260925 10:54:18.498497 1153 tablet_bootstrap.cc:654] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:18.499692 1153 log.cc:824] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:18.500411 1154 tablet_bootstrap.cc:492] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221: No bootstrap required, opened a new log
I20260925 10:54:18.500674 1154 ts_tablet_manager.cc:1397] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260925 10:54:18.502888 1153 tablet_bootstrap.cc:492] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447: No bootstrap required, opened a new log
I20260925 10:54:18.503140 1153 ts_tablet_manager.cc:1397] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.006s
I20260925 10:54:18.508062 1154 raft_consensus.cc:348] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.508442 1154 raft_consensus.cc:374] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:18.508530 1154 raft_consensus.cc:729] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b58aaebaa14049a393c051bf16e5a221, State: Initialized, Role: FOLLOWER
I20260925 10:54:18.508867 1152 raft_consensus.cc:348] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.509284 1152 raft_consensus.cc:374] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:18.509130 1154 consensus_queue.cc:260] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.509374 1152 raft_consensus.cc:729] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ec3d8f228746c3b3c8116be6aaeca3, State: Initialized, Role: FOLLOWER
I20260925 10:54:18.510025 1152 consensus_queue.cc:260] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.510214 1133 heartbeater.cc:503] Master 127.31.62.190:40663 was elected leader, sending a full tablet report...
I20260925 10:54:18.510955 1154 ts_tablet_manager.cc:1428] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260925 10:54:18.511356 994 heartbeater.cc:503] Master 127.31.62.190:40663 was elected leader, sending a full tablet report...
I20260925 10:54:18.511668 1152 ts_tablet_manager.cc:1428] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260925 10:54:18.514756 1153 raft_consensus.cc:348] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.515125 1153 raft_consensus.cc:374] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:18.515229 1153 raft_consensus.cc:729] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 387792af2d024535994dd2bc9b75c447, State: Initialized, Role: FOLLOWER
I20260925 10:54:18.515865 1153 consensus_queue.cc:260] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.517602 1153 ts_tablet_manager.cc:1428] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260925 10:54:18.519129 1160 raft_consensus.cc:482] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:54:18.519385 1160 raft_consensus.cc:504] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.521127 1160 leader_election.cc:288] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00ec3d8f228746c3b3c8116be6aaeca3 (127.31.62.130:46163), b58aaebaa14049a393c051bf16e5a221 (127.31.62.131:36413)
I20260925 10:54:18.527381 1150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.522019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:45617 (local address 127.31.62.131:36413)
0925 10:54:18.522191 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:18.522197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:18.522214 (+ 17us) server_negotiation.cc:407] Connection header received
0925 10:54:18.522340 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:18.522345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:18.522407 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:18.522475 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:18.523003 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.523691 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.525611 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.526509 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.526671 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.526746 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:18.527016 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 10:54:18.527136 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 10:54:18.527500 1147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.521846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:36345 (local address 127.31.62.130:46163)
0925 10:54:18.522011 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:18.522017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:18.522088 (+ 71us) server_negotiation.cc:407] Connection header received
0925 10:54:18.522408 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:18.522413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:18.522471 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:18.522517 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:18.523158 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.523856 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.525764 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.526635 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.526780 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.526850 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:18.527133 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 10:54:18.527256 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 10:54:18.527518 1162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.521377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:36413 (local address 127.31.62.129:45617)
0925 10:54:18.522085 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:18.522102 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:18.522228 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:18.522497 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:18.522501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:18.522522 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:18.522869 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.522878 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.523803 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.523806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.525474 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.525486 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.526700 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.526706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.526862 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.526904 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 10:54:18.526980 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 10:54:18.527124 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":6,"thread_start_us":637,"threads_started":1}
I20260925 10:54:18.527887 1161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.521315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:46163 (local address 127.31.62.129:36345)
0925 10:54:18.521995 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:18.522015 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:18.522258 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:18.522666 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:18.522669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:18.522686 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:18.523030 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.523037 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.523984 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.523988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.525630 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.525642 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.526840 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.526846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.526997 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.527019 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 10:54:18.527093 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 10:54:18.527226 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":40,"thread_start_us":293,"threads_started":1}
I20260925 10:54:18.528388 1083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753da43281cb4e058e147774f6bfd202" candidate_uuid: "387792af2d024535994dd2bc9b75c447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b58aaebaa14049a393c051bf16e5a221" is_pre_election: true
I20260925 10:54:18.528702 944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753da43281cb4e058e147774f6bfd202" candidate_uuid: "387792af2d024535994dd2bc9b75c447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" is_pre_election: true
I20260925 10:54:18.528879 1083 raft_consensus.cc:2454] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 387792af2d024535994dd2bc9b75c447 in term 0.
I20260925 10:54:18.529158 944 raft_consensus.cc:2454] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 387792af2d024535994dd2bc9b75c447 in term 0.
I20260925 10:54:18.529496 736 leader_election.cc:302] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 387792af2d024535994dd2bc9b75c447, b58aaebaa14049a393c051bf16e5a221; no voters:
I20260925 10:54:18.529829 1160 raft_consensus.cc:2781] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:54:18.529970 1160 raft_consensus.cc:482] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:54:18.530062 1160 raft_consensus.cc:3037] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:18.532097 1160 raft_consensus.cc:504] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.532665 1160 leader_election.cc:288] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [CANDIDATE]: Term 1 election: Requested vote from peers 00ec3d8f228746c3b3c8116be6aaeca3 (127.31.62.130:46163), b58aaebaa14049a393c051bf16e5a221 (127.31.62.131:36413)
I20260925 10:54:18.533119 1083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753da43281cb4e058e147774f6bfd202" candidate_uuid: "387792af2d024535994dd2bc9b75c447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b58aaebaa14049a393c051bf16e5a221"
I20260925 10:54:18.533066 944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753da43281cb4e058e147774f6bfd202" candidate_uuid: "387792af2d024535994dd2bc9b75c447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00ec3d8f228746c3b3c8116be6aaeca3"
I20260925 10:54:18.533283 1083 raft_consensus.cc:3037] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:18.533321 944 raft_consensus.cc:3037] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:18.535561 1083 raft_consensus.cc:2454] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 387792af2d024535994dd2bc9b75c447 in term 1.
I20260925 10:54:18.535590 944 raft_consensus.cc:2454] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 387792af2d024535994dd2bc9b75c447 in term 1.
I20260925 10:54:18.535929 736 leader_election.cc:302] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 387792af2d024535994dd2bc9b75c447, b58aaebaa14049a393c051bf16e5a221; no voters:
I20260925 10:54:18.536206 1160 raft_consensus.cc:2781] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:18.536669 1160 raft_consensus.cc:686] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [term 1 LEADER]: Becoming Leader. State: Replica: 387792af2d024535994dd2bc9b75c447, State: Running, Role: LEADER
I20260925 10:54:18.537190 1160 consensus_queue.cc:237] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.541958 647 catalog_manager.cc:5495] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 reported cstate change: term changed from 0 to 1, leader changed from <none> to 387792af2d024535994dd2bc9b75c447 (127.31.62.129). New cstate: current_term: 1 leader_uuid: "387792af2d024535994dd2bc9b75c447" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } health_report { overall_health: UNKNOWN } } }
I20260925 10:54:18.581314 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.575104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35114 (local address 127.31.62.129:44159)
0925 10:54:18.575303 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:18.575307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:18.575496 (+ 189us) server_negotiation.cc:407] Connection header received
0925 10:54:18.575770 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:18.575774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:18.575872 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:18.575999 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:54:18.576438 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.577308 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.578737 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.579217 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.579323 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.579956 (+ 633us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:54:18.580582 (+ 626us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:54:18.580743 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:18.580951 (+ 208us) server_negotiation.cc:299] Negotiation successful
0925 10:54:18.581039 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260925 10:54:18.589908 944 raft_consensus.cc:1261] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 1 FOLLOWER]: Refusing update from remote peer 387792af2d024535994dd2bc9b75c447: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:18.589907 1083 raft_consensus.cc:1261] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221 [term 1 FOLLOWER]: Refusing update from remote peer 387792af2d024535994dd2bc9b75c447: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:18.591167 1160 consensus_queue.cc:1035] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:18.591480 1163 consensus_queue.cc:1035] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:18.603048 1167 mvcc.cc:205] Tried to move back new op lower bound from 7333206665572966400 to 7333206665372237824. Current Snapshot: MvccSnapshot[applied={T|T < 7333206665572966400}]
I20260925 10:54:18.608384 1168 mvcc.cc:205] Tried to move back new op lower bound from 7333206665572966400 to 7333206665372237824. Current Snapshot: MvccSnapshot[applied={T|T < 7333206665572966400}]
I20260925 10:54:18.627063 648 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:18.627372 648 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:18.631532 648 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79: Sending DeleteTablet for 3 replicas of tablet 753da43281cb4e058e147774f6bfd202
I20260925 10:54:18.632720 924 tablet_service.cc:1514] Processing DeleteTablet for tablet 753da43281cb4e058e147774f6bfd202 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:18 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:51914
I20260925 10:54:18.632740 1063 tablet_service.cc:1514] Processing DeleteTablet for tablet 753da43281cb4e058e147774f6bfd202 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:18 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:36170
I20260925 10:54:18.632893 785 tablet_service.cc:1514] Processing DeleteTablet for tablet 753da43281cb4e058e147774f6bfd202 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:18 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:35112
I20260925 10:54:18.636283 648 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:54:18.637120 648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:54:18.647971 1062 tablet_service.cc:1467] Processing CreateTablet for tablet 9b071440ca8d4be1839bc23129fc93f5 (DEFAULT_TABLE table=test-table [id=3c82349909184dd889b75fdbaf129652]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:18.648718 1062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b071440ca8d4be1839bc23129fc93f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:18.650951 1177 tablet_replica.cc:330] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221: stopping tablet replica
I20260925 10:54:18.651075 1176 tablet_replica.cc:330] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3: stopping tablet replica
I20260925 10:54:18.651504 1177 raft_consensus.cc:2229] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:54:18.651505 1176 raft_consensus.cc:2229] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:54:18.651998 1176 raft_consensus.cc:2258] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:18.652096 1177 raft_consensus.cc:2258] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:18.653705 784 tablet_service.cc:1467] Processing CreateTablet for tablet 9b071440ca8d4be1839bc23129fc93f5 (DEFAULT_TABLE table=test-table [id=3c82349909184dd889b75fdbaf129652]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:18.654265 1177 ts_tablet_manager.cc:1905] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:18.654323 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b071440ca8d4be1839bc23129fc93f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:18.657424 924 tablet_service.cc:1467] Processing CreateTablet for tablet 9b071440ca8d4be1839bc23129fc93f5 (DEFAULT_TABLE table=test-table [id=3c82349909184dd889b75fdbaf129652]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:18.657864 924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b071440ca8d4be1839bc23129fc93f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:18.660794 1153 tablet_bootstrap.cc:492] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447: Bootstrap starting.
I20260925 10:54:18.662542 1153 tablet_bootstrap.cc:654] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:18.662871 1154 tablet_bootstrap.cc:492] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221: Bootstrap starting.
I20260925 10:54:18.664058 1176 ts_tablet_manager.cc:1905] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:18.664389 1154 tablet_bootstrap.cc:654] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:18.668643 1177 ts_tablet_manager.cc:1918] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:54:18.669063 1177 log.cc:1196] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/753da43281cb4e058e147774f6bfd202
I20260925 10:54:18.669852 1177 ts_tablet_manager.cc:1939] T 753da43281cb4e058e147774f6bfd202 P b58aaebaa14049a393c051bf16e5a221: Deleting consensus metadata
I20260925 10:54:18.672297 1152 tablet_bootstrap.cc:492] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3: Bootstrap starting.
I20260925 10:54:18.672446 633 catalog_manager.cc:4841] TS b58aaebaa14049a393c051bf16e5a221 (127.31.62.131:36413): tablet 753da43281cb4e058e147774f6bfd202 (table test-table [id=fae6daa801884b788aef4f1c13fac292]) successfully deleted
I20260925 10:54:18.673561 1154 tablet_bootstrap.cc:492] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221: No bootstrap required, opened a new log
I20260925 10:54:18.674031 1152 tablet_bootstrap.cc:654] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:18.673717 1154 ts_tablet_manager.cc:1397] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260925 10:54:18.675681 1178 tablet_replica.cc:330] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447: stopping tablet replica
I20260925 10:54:18.675806 1154 raft_consensus.cc:348] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.676011 1154 raft_consensus.cc:374] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:18.676102 1154 raft_consensus.cc:729] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b58aaebaa14049a393c051bf16e5a221, State: Initialized, Role: FOLLOWER
I20260925 10:54:18.676213 1153 tablet_bootstrap.cc:492] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447: No bootstrap required, opened a new log
I20260925 10:54:18.676169 1178 raft_consensus.cc:2229] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:54:18.676461 1153 ts_tablet_manager.cc:1397] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.004s
I20260925 10:54:18.676403 1154 consensus_queue.cc:260] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.676940 1178 raft_consensus.cc:2258] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:18.677001 1154 ts_tablet_manager.cc:1428] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 10:54:18.677672 1153 raft_consensus.cc:348] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.677938 1153 raft_consensus.cc:374] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:18.678104 1153 raft_consensus.cc:729] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 387792af2d024535994dd2bc9b75c447, State: Initialized, Role: FOLLOWER
I20260925 10:54:18.678408 1153 consensus_queue.cc:260] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.679708 1153 ts_tablet_manager.cc:1428] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 10:54:18.681408 1176 ts_tablet_manager.cc:1918] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:54:18.681602 1176 log.cc:1196] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/753da43281cb4e058e147774f6bfd202
I20260925 10:54:18.682142 1176 ts_tablet_manager.cc:1939] T 753da43281cb4e058e147774f6bfd202 P 00ec3d8f228746c3b3c8116be6aaeca3: Deleting consensus metadata
I20260925 10:54:18.684026 632 catalog_manager.cc:4841] TS 00ec3d8f228746c3b3c8116be6aaeca3 (127.31.62.130:46163): tablet 753da43281cb4e058e147774f6bfd202 (table test-table [id=fae6daa801884b788aef4f1c13fac292]) successfully deleted
I20260925 10:54:18.684926 1178 ts_tablet_manager.cc:1905] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:18.685817 1152 tablet_bootstrap.cc:492] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3: No bootstrap required, opened a new log
I20260925 10:54:18.685959 1152 ts_tablet_manager.cc:1397] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.005s
I20260925 10:54:18.687095 1152 raft_consensus.cc:348] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.687320 1152 raft_consensus.cc:374] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:18.687413 1152 raft_consensus.cc:729] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ec3d8f228746c3b3c8116be6aaeca3, State: Initialized, Role: FOLLOWER
I20260925 10:54:18.687682 1152 consensus_queue.cc:260] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.688303 1152 ts_tablet_manager.cc:1428] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260925 10:54:18.692624 1178 ts_tablet_manager.cc:1918] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:54:18.692821 1178 log.cc:1196] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/753da43281cb4e058e147774f6bfd202
I20260925 10:54:18.693414 1178 ts_tablet_manager.cc:1939] T 753da43281cb4e058e147774f6bfd202 P 387792af2d024535994dd2bc9b75c447: Deleting consensus metadata
I20260925 10:54:18.695160 634 catalog_manager.cc:4841] TS 387792af2d024535994dd2bc9b75c447 (127.31.62.129:44159): tablet 753da43281cb4e058e147774f6bfd202 (table test-table [id=fae6daa801884b788aef4f1c13fac292]) successfully deleted
I20260925 10:54:18.731271 1158 raft_consensus.cc:482] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:54:18.731529 1158 raft_consensus.cc:504] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.733570 1158 leader_election.cc:288] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 387792af2d024535994dd2bc9b75c447 (127.31.62.129:44159), 00ec3d8f228746c3b3c8116be6aaeca3 (127.31.62.130:46163)
I20260925 10:54:18.741137 1147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.734256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:58335 (local address 127.31.62.130:46163)
0925 10:54:18.734422 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:18.734428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:18.734663 (+ 235us) server_negotiation.cc:407] Connection header received
0925 10:54:18.734740 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:18.734746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:18.734822 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:18.734898 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:18.735963 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.737160 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.738977 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.740141 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.740313 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.740388 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:18.740711 (+ 323us) server_negotiation.cc:299] Negotiation successful
0925 10:54:18.740834 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 10:54:18.741170 1022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.734099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:46163 (local address 127.31.62.131:58335)
0925 10:54:18.734401 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:18.734421 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:18.734551 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:18.735054 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:18.735060 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:18.735086 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:18.735301 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.735313 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.737302 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.737307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.738763 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.738777 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.740322 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.740328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.740537 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.740581 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 10:54:18.740681 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 10:54:18.740821 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":4}
I20260925 10:54:18.742071 944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b071440ca8d4be1839bc23129fc93f5" candidate_uuid: "b58aaebaa14049a393c051bf16e5a221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" is_pre_election: true
I20260925 10:54:18.742436 944 raft_consensus.cc:2454] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b58aaebaa14049a393c051bf16e5a221 in term 0.
I20260925 10:54:18.742940 1014 leader_election.cc:302] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00ec3d8f228746c3b3c8116be6aaeca3, b58aaebaa14049a393c051bf16e5a221; no voters:
I20260925 10:54:18.743404 1158 raft_consensus.cc:2781] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:54:18.743496 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.735682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:56413 (local address 127.31.62.129:44159)
0925 10:54:18.735880 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:18.735885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:18.736787 (+ 902us) server_negotiation.cc:407] Connection header received
0925 10:54:18.736961 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:18.736966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:18.737023 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:18.737102 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:18.737791 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.739000 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.741323 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.742465 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.742618 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.742688 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:18.743055 (+ 367us) server_negotiation.cc:299] Negotiation successful
0925 10:54:18.743187 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 10:54:18.743530 1158 raft_consensus.cc:482] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:54:18.743613 1158 raft_consensus.cc:3037] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:18.743964 1182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.734815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:44159 (local address 127.31.62.131:56413)
0925 10:54:18.736670 (+ 1855us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:18.736686 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:18.736821 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:18.737138 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:18.737143 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:18.737168 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:18.737585 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.737600 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.739132 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.739138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.741043 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.741068 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.742694 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.742700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.742878 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.742911 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 10:54:18.743011 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 10:54:18.743252 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1745,"mutex_wait_us":1,"thread_start_us":1588,"threads_started":1}
I20260925 10:54:18.744889 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b071440ca8d4be1839bc23129fc93f5" candidate_uuid: "b58aaebaa14049a393c051bf16e5a221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "387792af2d024535994dd2bc9b75c447" is_pre_election: true
I20260925 10:54:18.745343 805 raft_consensus.cc:2454] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b58aaebaa14049a393c051bf16e5a221 in term 0.
I20260925 10:54:18.746467 1158 raft_consensus.cc:504] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.747242 1158 leader_election.cc:288] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [CANDIDATE]: Term 1 election: Requested vote from peers 387792af2d024535994dd2bc9b75c447 (127.31.62.129:44159), 00ec3d8f228746c3b3c8116be6aaeca3 (127.31.62.130:46163)
I20260925 10:54:18.747475 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b071440ca8d4be1839bc23129fc93f5" candidate_uuid: "b58aaebaa14049a393c051bf16e5a221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "387792af2d024535994dd2bc9b75c447"
I20260925 10:54:18.747579 944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b071440ca8d4be1839bc23129fc93f5" candidate_uuid: "b58aaebaa14049a393c051bf16e5a221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00ec3d8f228746c3b3c8116be6aaeca3"
I20260925 10:54:18.747690 805 raft_consensus.cc:3037] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:18.747778 944 raft_consensus.cc:3037] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:18.749874 944 raft_consensus.cc:2454] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b58aaebaa14049a393c051bf16e5a221 in term 1.
I20260925 10:54:18.750366 1014 leader_election.cc:302] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00ec3d8f228746c3b3c8116be6aaeca3, b58aaebaa14049a393c051bf16e5a221; no voters:
I20260925 10:54:18.750757 1158 raft_consensus.cc:2781] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:18.751003 805 raft_consensus.cc:2454] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b58aaebaa14049a393c051bf16e5a221 in term 1.
I20260925 10:54:18.760154 1158 raft_consensus.cc:686] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [term 1 LEADER]: Becoming Leader. State: Replica: b58aaebaa14049a393c051bf16e5a221, State: Running, Role: LEADER
I20260925 10:54:18.760630 1158 consensus_queue.cc:237] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.764240 646 catalog_manager.cc:5495] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 reported cstate change: term changed from 0 to 1, leader changed from <none> to b58aaebaa14049a393c051bf16e5a221 (127.31.62.131). New cstate: current_term: 1 leader_uuid: "b58aaebaa14049a393c051bf16e5a221" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } health_report { overall_health: HEALTHY } } }
Sep 25 10:54:18 dist-test-slave-nx7s krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333656, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:18.827256 1190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.811416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40663 (local address 127.0.0.1:53360)
0925 10:54:18.812087 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:18.812107 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:18.812247 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:18.812610 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:18.812614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:18.813420 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:18.813683 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.813693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.815161 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.815165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.816149 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.816161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.817172 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.817177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.817297 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.817965 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:18.817982 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:18.820488 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:18.824505 (+ 4017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:18.824519 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:18.824526 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:18.825113 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:18.825560 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:18.825564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:18.825566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:18.825717 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:18.826351 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:18.826358 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:18.826524 (+ 166us) client_negotiation.cc:769] Sending connection context
0925 10:54:18.826670 (+ 146us) client_negotiation.cc:240] Negotiation successful
0925 10:54:18.826945 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":386,"threads_started":1}
I20260925 10:54:18.827708 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.811442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53360 (local address 127.31.62.190:40663)
0925 10:54:18.811633 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:18.811640 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:18.812213 (+ 573us) server_negotiation.cc:407] Connection header received
0925 10:54:18.812355 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:18.812361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:18.812461 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:18.812583 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:18.813850 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.815011 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.816290 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.817000 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.817126 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.820698 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:18.820705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:18.820711 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:18.820752 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:18.824277 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:18.825258 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:18.825264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:18.825267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:18.825444 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:18.825828 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:18.825833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:18.825837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:18.826209 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:18.826352 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:18.826801 (+ 449us) server_negotiation.cc:299] Negotiation successful
0925 10:54:18.827117 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":66}
I20260925 10:54:18.833653 646 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:53360:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 10:54:18.835072 646 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:54:18.842861 785 tablet_service.cc:1467] Processing CreateTablet for tablet 37d2d09b7a0b4df9a024413543d56f1f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8086a0c01acd4a1986e7df8a90f5aee8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 10:54:18.842902 924 tablet_service.cc:1467] Processing CreateTablet for tablet 37d2d09b7a0b4df9a024413543d56f1f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8086a0c01acd4a1986e7df8a90f5aee8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 10:54:18.843492 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37d2d09b7a0b4df9a024413543d56f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:18.843490 924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37d2d09b7a0b4df9a024413543d56f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:18.844491 1062 tablet_service.cc:1467] Processing CreateTablet for tablet 37d2d09b7a0b4df9a024413543d56f1f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8086a0c01acd4a1986e7df8a90f5aee8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 10:54:18.845090 1062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37d2d09b7a0b4df9a024413543d56f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:18.850857 1152 tablet_bootstrap.cc:492] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3: Bootstrap starting.
I20260925 10:54:18.851078 1153 tablet_bootstrap.cc:492] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447: Bootstrap starting.
I20260925 10:54:18.852672 1152 tablet_bootstrap.cc:654] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:18.852757 1153 tablet_bootstrap.cc:654] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447: Neither blocks nor log segments found. Creating new log.
W20260925 10:54:18.853920 856 tablet.cc:2369] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:54:18.857494 1154 tablet_bootstrap.cc:492] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221: Bootstrap starting.
I20260925 10:54:18.859427 1154 tablet_bootstrap.cc:654] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221: Neither blocks nor log segments found. Creating new log.
W20260925 10:54:18.863165 995 tablet.cc:2369] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:54:18.863615 1154 tablet_bootstrap.cc:492] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221: No bootstrap required, opened a new log
I20260925 10:54:18.863619 1152 tablet_bootstrap.cc:492] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3: No bootstrap required, opened a new log
I20260925 10:54:18.863778 1154 ts_tablet_manager.cc:1397] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260925 10:54:18.863782 1152 ts_tablet_manager.cc:1397] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260925 10:54:18.864504 1154 raft_consensus.cc:348] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.864508 1152 raft_consensus.cc:348] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.864751 1152 raft_consensus.cc:374] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:18.864753 1154 raft_consensus.cc:374] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:18.864867 1152 raft_consensus.cc:729] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ec3d8f228746c3b3c8116be6aaeca3, State: Initialized, Role: FOLLOWER
I20260925 10:54:18.864877 1154 raft_consensus.cc:729] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b58aaebaa14049a393c051bf16e5a221, State: Initialized, Role: FOLLOWER
I20260925 10:54:18.865053 1153 tablet_bootstrap.cc:492] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447: No bootstrap required, opened a new log
I20260925 10:54:18.865214 1153 ts_tablet_manager.cc:1397] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260925 10:54:18.865121 1154 consensus_queue.cc:260] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.865137 1152 consensus_queue.cc:260] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.865667 1154 ts_tablet_manager.cc:1428] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 10:54:18.865687 1152 ts_tablet_manager.cc:1428] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 10:54:18.865928 1153 raft_consensus.cc:348] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
W20260925 10:54:18.866020 1134 tablet.cc:2369] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:54:18.866149 1153 raft_consensus.cc:374] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:18.866206 1153 raft_consensus.cc:729] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 387792af2d024535994dd2bc9b75c447, State: Initialized, Role: FOLLOWER
I20260925 10:54:18.866400 1153 consensus_queue.cc:260] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.866605 1159 tablet_replica.cc:439] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } } }
I20260925 10:54:18.866981 1159 tablet_replica.cc:442] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:54:18.866832 1174 tablet_replica.cc:439] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } } }
I20260925 10:54:18.867159 1174 tablet_replica.cc:442] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:54:18.868155 1158 tablet_replica.cc:439] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } } }
I20260925 10:54:18.868422 1158 tablet_replica.cc:442] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:54:18.869838 1153 ts_tablet_manager.cc:1428] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260925 10:54:18.952955 1158 raft_consensus.cc:482] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:54:18.953262 1158 raft_consensus.cc:504] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.953939 1158 leader_election.cc:288] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 387792af2d024535994dd2bc9b75c447 (127.31.62.129:44159), 00ec3d8f228746c3b3c8116be6aaeca3 (127.31.62.130:46163)
I20260925 10:54:18.954540 944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37d2d09b7a0b4df9a024413543d56f1f" candidate_uuid: "b58aaebaa14049a393c051bf16e5a221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" is_pre_election: true
I20260925 10:54:18.954579 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37d2d09b7a0b4df9a024413543d56f1f" candidate_uuid: "b58aaebaa14049a393c051bf16e5a221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "387792af2d024535994dd2bc9b75c447" is_pre_election: true
I20260925 10:54:18.954825 944 raft_consensus.cc:2454] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b58aaebaa14049a393c051bf16e5a221 in term 0.
I20260925 10:54:18.954835 805 raft_consensus.cc:2454] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b58aaebaa14049a393c051bf16e5a221 in term 0.
I20260925 10:54:18.955343 1016 leader_election.cc:302] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 00ec3d8f228746c3b3c8116be6aaeca3, 387792af2d024535994dd2bc9b75c447, b58aaebaa14049a393c051bf16e5a221; no voters:
I20260925 10:54:18.955646 1158 raft_consensus.cc:2781] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:54:18.955780 1158 raft_consensus.cc:482] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:54:18.955896 1158 raft_consensus.cc:3037] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:18.959041 1158 raft_consensus.cc:504] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.959860 1158 leader_election.cc:288] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [CANDIDATE]: Term 1 election: Requested vote from peers 387792af2d024535994dd2bc9b75c447 (127.31.62.129:44159), 00ec3d8f228746c3b3c8116be6aaeca3 (127.31.62.130:46163)
I20260925 10:54:18.960342 944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37d2d09b7a0b4df9a024413543d56f1f" candidate_uuid: "b58aaebaa14049a393c051bf16e5a221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00ec3d8f228746c3b3c8116be6aaeca3"
I20260925 10:54:18.960369 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37d2d09b7a0b4df9a024413543d56f1f" candidate_uuid: "b58aaebaa14049a393c051bf16e5a221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "387792af2d024535994dd2bc9b75c447"
I20260925 10:54:18.960582 944 raft_consensus.cc:3037] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:18.960613 805 raft_consensus.cc:3037] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:18.962834 944 raft_consensus.cc:2454] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b58aaebaa14049a393c051bf16e5a221 in term 1.
I20260925 10:54:18.963357 1014 leader_election.cc:302] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00ec3d8f228746c3b3c8116be6aaeca3, b58aaebaa14049a393c051bf16e5a221; no voters:
I20260925 10:54:18.963632 1158 raft_consensus.cc:2781] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:18.963832 1158 raft_consensus.cc:686] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [term 1 LEADER]: Becoming Leader. State: Replica: b58aaebaa14049a393c051bf16e5a221, State: Running, Role: LEADER
I20260925 10:54:18.964088 1158 consensus_queue.cc:237] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } }
I20260925 10:54:18.964141 805 raft_consensus.cc:2454] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b58aaebaa14049a393c051bf16e5a221 in term 1.
I20260925 10:54:18.965853 1183 tablet_replica.cc:439] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221: TxnStatusTablet state changed. Reason: New leader b58aaebaa14049a393c051bf16e5a221. Latest consensus state: current_term: 1 leader_uuid: "b58aaebaa14049a393c051bf16e5a221" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } } }
I20260925 10:54:18.966202 1183 tablet_replica.cc:442] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221: This TxnStatusTablet replica's current role is: LEADER
I20260925 10:54:18.967264 1195 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 10:54:18.967506 1195 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 10:54:18.967939 648 catalog_manager.cc:5495] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 reported cstate change: term changed from 0 to 1, leader changed from <none> to b58aaebaa14049a393c051bf16e5a221 (127.31.62.131). New cstate: current_term: 1 leader_uuid: "b58aaebaa14049a393c051bf16e5a221" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } health_report { overall_health: HEALTHY } } }
I20260925 10:54:18.996604 1150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.990764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36180 (local address 127.31.62.131:36413)
0925 10:54:18.990930 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:18.990936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:18.991139 (+ 203us) server_negotiation.cc:407] Connection header received
0925 10:54:18.991493 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:18.991497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:18.991571 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:18.991686 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:54:18.992035 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.993124 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.994406 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.994827 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.994940 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.995396 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:54:18.995957 (+ 561us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:54:18.996104 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:18.996191 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 10:54:18.996274 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 10:54:18.996614 1190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:18.990703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:36413 (local address 127.0.0.1:36180)
0925 10:54:18.991043 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:18.991080 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:18.991226 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:18.991726 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:18.991730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:18.991743 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 10:54:18.991911 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.991919 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.993258 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.993264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.994257 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:18.994269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:18.994978 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:18.994984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:18.995132 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:18.995192 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 10:54:18.996067 (+ 875us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 10:54:18.996094 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 10:54:18.996172 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 10:54:18.996342 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184}
I20260925 10:54:19.000595 944 raft_consensus.cc:1261] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3 [term 1 FOLLOWER]: Refusing update from remote peer b58aaebaa14049a393c051bf16e5a221: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:19.000813 805 raft_consensus.cc:1261] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447 [term 1 FOLLOWER]: Refusing update from remote peer b58aaebaa14049a393c051bf16e5a221: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:19.001453 1158 consensus_queue.cc:1035] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:19.001816 1183 consensus_queue.cc:1035] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221 [LEADER]: Connected to new peer: Peer: permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:19.016116 1159 tablet_replica.cc:439] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3: TxnStatusTablet state changed. Reason: New leader b58aaebaa14049a393c051bf16e5a221. Latest consensus state: current_term: 1 leader_uuid: "b58aaebaa14049a393c051bf16e5a221" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } } }
I20260925 10:54:19.016448 1159 tablet_replica.cc:442] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:54:19.016642 1174 tablet_replica.cc:439] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447: TxnStatusTablet state changed. Reason: New leader b58aaebaa14049a393c051bf16e5a221. Latest consensus state: current_term: 1 leader_uuid: "b58aaebaa14049a393c051bf16e5a221" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } } }
I20260925 10:54:19.016999 1174 tablet_replica.cc:442] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:54:19.018085 1183 tablet_replica.cc:439] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b58aaebaa14049a393c051bf16e5a221" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } } }
I20260925 10:54:19.018368 1183 tablet_replica.cc:442] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221: This TxnStatusTablet replica's current role is: LEADER
I20260925 10:54:19.021166 1158 tablet_replica.cc:439] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b58aaebaa14049a393c051bf16e5a221" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } } }
I20260925 10:54:19.021445 1158 tablet_replica.cc:442] T 37d2d09b7a0b4df9a024413543d56f1f P b58aaebaa14049a393c051bf16e5a221: This TxnStatusTablet replica's current role is: LEADER
I20260925 10:54:19.021842 1174 tablet_replica.cc:439] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b58aaebaa14049a393c051bf16e5a221" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } } }
I20260925 10:54:19.022069 1174 tablet_replica.cc:442] T 37d2d09b7a0b4df9a024413543d56f1f P 387792af2d024535994dd2bc9b75c447: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:54:19.021950 1166 mvcc.cc:205] Tried to move back new op lower bound from 7333206667258728448 to 7333206667119509504. Current Snapshot: MvccSnapshot[applied={T|T < 7333206667258728448}]
I20260925 10:54:19.024268 1159 tablet_replica.cc:439] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b58aaebaa14049a393c051bf16e5a221" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 } } peers { permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 } } peers { permanent_uuid: "b58aaebaa14049a393c051bf16e5a221" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36413 } } }
I20260925 10:54:19.024489 1159 tablet_replica.cc:442] T 37d2d09b7a0b4df9a024413543d56f1f P 00ec3d8f228746c3b3c8116be6aaeca3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:54:19.031572 1150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:19.025878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36196 (local address 127.31.62.131:36413)
0925 10:54:19.026041 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:19.026046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:19.026064 (+ 18us) server_negotiation.cc:407] Connection header received
0925 10:54:19.026259 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:19.026264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:19.026345 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:19.026434 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:54:19.027008 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.027919 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.029421 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.030043 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.030188 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:19.030847 (+ 659us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:54:19.030977 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:54:19.031051 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:19.031208 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 10:54:19.031285 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 10:54:19.048385 1150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:19.042728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:51995 (local address 127.31.62.131:36413)
0925 10:54:19.042951 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:19.042957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:19.043320 (+ 363us) server_negotiation.cc:407] Connection header received
0925 10:54:19.043500 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:19.043505 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:19.043569 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:19.043640 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:54:19.044127 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.044968 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.046387 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.046920 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.047020 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:19.047502 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:54:19.047650 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:54:19.047720 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:19.047911 (+ 191us) server_negotiation.cc:299] Negotiation successful
0925 10:54:19.047995 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260925 10:54:19.048421 1202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:19.042673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:36413 (local address 127.31.62.131:51995)
0925 10:54:19.043222 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:19.043238 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:19.043383 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:19.043660 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:19.043664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:19.043675 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 10:54:19.043986 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:19.043995 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.045083 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.045088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:19.046259 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:19.046272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.047103 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.047110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:19.047290 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:19.047349 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 10:54:19.047734 (+ 385us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 10:54:19.047743 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 10:54:19.047890 (+ 147us) client_negotiation.cc:240] Negotiation successful
0925 10:54:19.048052 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":4,"thread_start_us":277,"threads_started":1}
I20260925 10:54:19.059648 805 raft_consensus.cc:1261] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447 [term 1 FOLLOWER]: Refusing update from remote peer b58aaebaa14049a393c051bf16e5a221: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:19.060256 944 raft_consensus.cc:1261] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 1 FOLLOWER]: Refusing update from remote peer b58aaebaa14049a393c051bf16e5a221: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:19.060256 1158 consensus_queue.cc:1035] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [LEADER]: Connected to new peer: Peer: permanent_uuid: "387792af2d024535994dd2bc9b75c447" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:19.061285 1158 consensus_queue.cc:1035] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00ec3d8f228746c3b3c8116be6aaeca3" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 46163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:19.146842 645 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:19.147099 645 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:19.150040 645 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8967a1520d804688a34036c4526d9b79: Sending DeleteTablet for 3 replicas of tablet 9b071440ca8d4be1839bc23129fc93f5
I20260925 10:54:19.150847 785 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b071440ca8d4be1839bc23129fc93f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:19 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:35112
I20260925 10:54:19.151127 1178 tablet_replica.cc:330] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447: stopping tablet replica
I20260925 10:54:19.151124 924 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b071440ca8d4be1839bc23129fc93f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:19 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:51914
I20260925 10:54:19.151310 1178 raft_consensus.cc:2229] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:54:19.151408 1176 tablet_replica.cc:330] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3: stopping tablet replica
I20260925 10:54:19.151489 1178 raft_consensus.cc:2258] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:19.151584 1176 raft_consensus.cc:2229] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:54:19.151765 1176 raft_consensus.cc:2258] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:19.152822 1178 ts_tablet_manager.cc:1905] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:19.153381 1176 ts_tablet_manager.cc:1905] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:19.154569 1062 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b071440ca8d4be1839bc23129fc93f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:19 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:36170
I20260925 10:54:19.154884 1177 tablet_replica.cc:330] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221: stopping tablet replica
I20260925 10:54:19.155082 1177 raft_consensus.cc:2229] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:54:19.155612 1177 raft_consensus.cc:2258] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:19.157637 1177 ts_tablet_manager.cc:1905] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:19.159894 1178 ts_tablet_manager.cc:1918] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 10:54:19.160058 1178 log.cc:1196] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/9b071440ca8d4be1839bc23129fc93f5
I20260925 10:54:19.160480 1178 ts_tablet_manager.cc:1939] T 9b071440ca8d4be1839bc23129fc93f5 P 387792af2d024535994dd2bc9b75c447: Deleting consensus metadata
I20260925 10:54:19.166368 634 catalog_manager.cc:4841] TS 387792af2d024535994dd2bc9b75c447 (127.31.62.129:44159): tablet 9b071440ca8d4be1839bc23129fc93f5 (table test-table [id=3c82349909184dd889b75fdbaf129652]) successfully deleted
I20260925 10:54:19.169435 1177 ts_tablet_manager.cc:1918] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 10:54:19.169624 1177 log.cc:1196] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/9b071440ca8d4be1839bc23129fc93f5
I20260925 10:54:19.170096 1177 ts_tablet_manager.cc:1939] T 9b071440ca8d4be1839bc23129fc93f5 P b58aaebaa14049a393c051bf16e5a221: Deleting consensus metadata
I20260925 10:54:19.171419 633 catalog_manager.cc:4841] TS b58aaebaa14049a393c051bf16e5a221 (127.31.62.131:36413): tablet 9b071440ca8d4be1839bc23129fc93f5 (table test-table [id=3c82349909184dd889b75fdbaf129652]) successfully deleted
I20260925 10:54:19.171980 1176 ts_tablet_manager.cc:1918] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 10:54:19.172138 1176 log.cc:1196] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/9b071440ca8d4be1839bc23129fc93f5
I20260925 10:54:19.172509 1176 ts_tablet_manager.cc:1939] T 9b071440ca8d4be1839bc23129fc93f5 P 00ec3d8f228746c3b3c8116be6aaeca3: Deleting consensus metadata
I20260925 10:54:19.174139 632 catalog_manager.cc:4841] TS 00ec3d8f228746c3b3c8116be6aaeca3 (127.31.62.130:46163): tablet 9b071440ca8d4be1839bc23129fc93f5 (table test-table [id=3c82349909184dd889b75fdbaf129652]) successfully deleted
Sep 25 10:54:19 dist-test-slave-nx7s krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:54:19.191409 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:19.167163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35126 (local address 127.31.62.129:44159)
0925 10:54:19.167403 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:19.167410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:19.174053 (+ 6643us) server_negotiation.cc:407] Connection header received
0925 10:54:19.174297 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:19.174304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:19.174400 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:19.174524 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:19.176141 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.177326 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.178733 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.179433 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.179558 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:19.183725 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:19.183746 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:19.183767 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:19.183815 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:19.188196 (+ 4381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:19.189156 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:19.189167 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:19.189170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:19.189367 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:19.189807 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:19.189811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:19.189815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:19.190253 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:19.190550 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:19.190812 (+ 262us) server_negotiation.cc:299] Negotiation successful
0925 10:54:19.191035 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
W20260925 10:54:19.192266 850 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35126
I20260925 10:54:19.209102 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:19.195442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53376 (local address 127.31.62.190:40663)
0925 10:54:19.195612 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:19.195617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:19.195818 (+ 201us) server_negotiation.cc:407] Connection header received
0925 10:54:19.195956 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:19.195962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:19.196042 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:19.196145 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:19.197388 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.198496 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.199689 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.200318 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.200453 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:19.202863 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:19.202871 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:19.202877 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:19.202922 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:19.206372 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:19.207107 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:19.207113 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:19.207116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:19.207269 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:19.207630 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:19.207635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:19.207638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:19.207981 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:19.208224 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:19.208503 (+ 279us) server_negotiation.cc:299] Negotiation successful
0925 10:54:19.208724 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260925 10:54:19.209828 645 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53376
I20260925 10:54:19.212174 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 719
I20260925 10:54:19.226364 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:19.213001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35134 (local address 127.31.62.129:44159)
0925 10:54:19.213168 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:19.213174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:19.213864 (+ 690us) server_negotiation.cc:407] Connection header received
0925 10:54:19.213980 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:19.213986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:19.214065 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:19.214166 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:19.215206 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.216379 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.217639 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.218268 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.218401 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:19.220567 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:19.220573 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:19.220577 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:19.220614 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:19.223715 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:19.224373 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:19.224379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:19.224383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:19.224529 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:19.224879 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:19.224885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:19.224888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:19.225220 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:19.225461 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:19.225713 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 10:54:19.225942 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260925 10:54:19.227052 850 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35134
W20260925 10:54:19.227634 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 719: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:54:19.227831 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 719
I20260925 10:54:19.245735 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 859
Sep 25 10:54:19 dist-test-slave-nx7s krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:54:19.263396 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:19.246453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51918 (local address 127.31.62.130:46163)
0925 10:54:19.246931 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:19.246938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:19.246957 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:54:19.247019 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:19.247025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:19.247111 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:19.247224 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:19.248477 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.249640 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.250979 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.251582 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.251700 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:19.256299 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:19.256318 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:19.256337 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:19.256387 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:19.260101 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:19.260990 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:19.261000 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:19.261004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:19.261181 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:19.261681 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:19.261686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:19.261689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:19.262152 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:19.262451 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:19.262747 (+ 296us) server_negotiation.cc:299] Negotiation successful
0925 10:54:19.262984 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":232,"threads_started":1}
W20260925 10:54:19.264338 989 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51918
W20260925 10:54:19.265058 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 859: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:54:19.265230 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 859
I20260925 10:54:19.279836 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 998
Sep 25 10:54:19 dist-test-slave-nx7s krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:54:19.295827 1150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:19.280735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36206 (local address 127.31.62.131:36413)
0925 10:54:19.280946 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:19.280952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:19.281023 (+ 71us) server_negotiation.cc:407] Connection header received
0925 10:54:19.281191 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:19.281197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:19.281274 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:19.281354 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:19.282572 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.283349 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.284590 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:19.285124 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:19.285291 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:19.289688 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:19.289702 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:19.289717 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:19.289771 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:19.292994 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:19.293724 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:19.293733 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:19.293736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:19.293889 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:19.294276 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:19.294280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:19.294282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:19.294638 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:19.294952 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:19.295246 (+ 294us) server_negotiation.cc:299] Negotiation successful
0925 10:54:19.295465 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260925 10:54:19.296541 1128 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36206
W20260925 10:54:19.297459 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 998: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:54:19.297632 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 998
I20260925 10:54:19.313050 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 616
I20260925 10:54:19.314105 688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:19.426821 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 616
2026-09-25T10:54:19Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4303 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:54:19 dist-test-slave-nx7s krb5kdc[1216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:54:19 dist-test-slave-nx7s krb5kdc[1216](info): set up 2 sockets
Sep 25 10:54:19 dist-test-slave-nx7s krb5kdc[1216](info): commencing operation
krb5kdc: starting...
W20260925 10:54:20.421344 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:54:20 dist-test-slave-nx7s krb5kdc[1216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:54:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:54:20.600536 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:44035
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:44035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:20.915841 1229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:20.916147 1229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:20.916239 1229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:20.923563 1229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:20.923684 1229 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 10:54:20.923731 1229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:20.923813 1229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:20.923874 1229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:20.923959 1229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:54:20.934360 1229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:44035
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:44035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:20.935715 1229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:20.937143 1229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:20.949246 1235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:20.949307 1234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:20.949334 1237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:20.949903 1229 server_base.cc:1034] running on GCE node
I20260925 10:54:20.950778 1229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:20.952376 1229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:20.953639 1229 hybrid_clock.cc:648] HybridClock initialized: now 1790333660953550 us; error 59 us; skew 500 ppm
I20260925 10:54:20.954005 1229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:20 dist-test-slave-nx7s krb5kdc[1216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333660, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:20.957319 1229 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:54:20.958933 1229 webserver.cc:460] Webserver started at http://127.31.62.190:43627/ using document root <none> and password file <none>
I20260925 10:54:20.959678 1229 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:20.959794 1229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:20.960105 1229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:20.963249 1229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "c74bb658ce914e448f49031e2e079fa6"
format_stamp: "Formatted at 2026-09-25 10:54:20 on dist-test-slave-nx7s"
server_key: "6e4b4e86ad20eb83fc768f100c03aaab"
server_key_iv: "5466a0a342b3d97f22eeb4d3928586ea"
server_key_version: "encryptionkey@0"
I20260925 10:54:20.964048 1229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "c74bb658ce914e448f49031e2e079fa6"
format_stamp: "Formatted at 2026-09-25 10:54:20 on dist-test-slave-nx7s"
server_key: "6e4b4e86ad20eb83fc768f100c03aaab"
server_key_iv: "5466a0a342b3d97f22eeb4d3928586ea"
server_key_version: "encryptionkey@0"
I20260925 10:54:20.969542 1229 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260925 10:54:20.973241 1244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:20.974668 1229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260925 10:54:20.974864 1229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "c74bb658ce914e448f49031e2e079fa6"
format_stamp: "Formatted at 2026-09-25 10:54:20 on dist-test-slave-nx7s"
server_key: "6e4b4e86ad20eb83fc768f100c03aaab"
server_key_iv: "5466a0a342b3d97f22eeb4d3928586ea"
server_key_version: "encryptionkey@0"
I20260925 10:54:20.975071 1229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:20.985935 1229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:20.990016 1229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:20.990271 1229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:21.010895 1229 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:44035
I20260925 10:54:21.010921 1305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:44035 every 8 connection(s)
I20260925 10:54:21.012217 1229 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:21.015962 1306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:21.017251 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 1229
I20260925 10:54:21.017442 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:21.018055 31994 external_mini_cluster.cc:1428] Setting key 446164ac870ac1a9d65ca53a26298081
I20260925 10:54:21.029809 1306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6: Bootstrap starting.
Sep 25 10:54:21 dist-test-slave-nx7s krb5kdc[1216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:21.033788 1306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:21.034965 1306 log.cc:824] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:21.038570 1306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6: No bootstrap required, opened a new log
I20260925 10:54:21.041390 1309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:21.019967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55252 (local address 127.31.62.190:44035)
0925 10:54:21.020872 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:21.020889 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:21.020940 (+ 51us) server_negotiation.cc:407] Connection header received
0925 10:54:21.022470 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:21.022493 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:21.022916 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:21.023234 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:21.025006 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:21.026361 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:21.028055 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:21.028727 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:21.028909 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:21.033543 (+ 4634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:21.033552 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:21.033569 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:21.033614 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:21.037074 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:21.038039 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:21.038045 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:21.038051 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:21.038224 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:21.038627 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:21.038632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:21.038635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:21.039201 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:21.039431 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:21.039879 (+ 448us) server_negotiation.cc:299] Negotiation successful
0925 10:54:21.040216 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":429,"threads_started":1}
I20260925 10:54:21.046396 1306 raft_consensus.cc:348] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74bb658ce914e448f49031e2e079fa6" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44035 } }
I20260925 10:54:21.046764 1306 raft_consensus.cc:374] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:21.046864 1306 raft_consensus.cc:729] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c74bb658ce914e448f49031e2e079fa6, State: Initialized, Role: FOLLOWER
I20260925 10:54:21.047429 1306 consensus_queue.cc:260] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74bb658ce914e448f49031e2e079fa6" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44035 } }
I20260925 10:54:21.047649 1306 raft_consensus.cc:388] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:21.047765 1306 raft_consensus.cc:482] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:21.047914 1306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:21.050037 1306 raft_consensus.cc:504] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74bb658ce914e448f49031e2e079fa6" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44035 } }
I20260925 10:54:21.050535 1306 leader_election.cc:302] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c74bb658ce914e448f49031e2e079fa6; no voters:
I20260925 10:54:21.051048 1306 leader_election.cc:288] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:21.051175 1311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:21.051774 1311 raft_consensus.cc:686] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [term 1 LEADER]: Becoming Leader. State: Replica: c74bb658ce914e448f49031e2e079fa6, State: Running, Role: LEADER
I20260925 10:54:21.052273 1311 consensus_queue.cc:237] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74bb658ce914e448f49031e2e079fa6" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44035 } }
I20260925 10:54:21.053097 1306 sys_catalog.cc:564] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:21.054728 1312 sys_catalog.cc:455] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c74bb658ce914e448f49031e2e079fa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74bb658ce914e448f49031e2e079fa6" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44035 } } }
I20260925 10:54:21.055120 1312 sys_catalog.cc:458] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:21.055456 1313 sys_catalog.cc:455] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c74bb658ce914e448f49031e2e079fa6. Latest consensus state: current_term: 1 leader_uuid: "c74bb658ce914e448f49031e2e079fa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74bb658ce914e448f49031e2e079fa6" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44035 } } }
I20260925 10:54:21.055775 1313 sys_catalog.cc:458] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:21.057408 1317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:21.062247 1317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:21.071055 1317 catalog_manager.cc:1347] Generated new cluster ID: 0edfbebf5ec74c66b7e8a9ecc75a60e7
I20260925 10:54:21.071190 1317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:21.087644 1317 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:21.088973 1317 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:21.102291 1317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6: Generated new TSK 0
I20260925 10:54:21.103164 1317 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:54:21.148195 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:44035
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:21.475322 1332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:21.475618 1332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:21.475749 1332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:21.483377 1332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:21.483492 1332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:21.483603 1332 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 10:54:21.483669 1332 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 10:54:21.483773 1332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:54:21.493940 1332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.62.190:44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:21.495262 1332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:21.496657 1332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:21.508880 1337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:21.509658 1338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:21.510551 1341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:21.510759 1332 server_base.cc:1034] running on GCE node
I20260925 10:54:21.511420 1332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:21.512472 1332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:21.513674 1332 hybrid_clock.cc:648] HybridClock initialized: now 1790333661513622 us; error 41 us; skew 500 ppm
I20260925 10:54:21.513990 1332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:21 dist-test-slave-nx7s krb5kdc[1216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333661, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:21.517146 1332 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:54:21.518585 1332 webserver.cc:460] Webserver started at http://127.31.62.129:33315/ using document root <none> and password file <none>
I20260925 10:54:21.519277 1332 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:21.519382 1332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:21.519673 1332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:21.522572 1332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "eaefc446ad8a420cbb49ffda1e09a0be"
format_stamp: "Formatted at 2026-09-25 10:54:21 on dist-test-slave-nx7s"
server_key: "b0d550783cf71404effa80f656567ee8"
server_key_iv: "cc29d1474c4f41b6d159845fe018d8df"
server_key_version: "encryptionkey@0"
I20260925 10:54:21.523330 1332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "eaefc446ad8a420cbb49ffda1e09a0be"
format_stamp: "Formatted at 2026-09-25 10:54:21 on dist-test-slave-nx7s"
server_key: "b0d550783cf71404effa80f656567ee8"
server_key_iv: "cc29d1474c4f41b6d159845fe018d8df"
server_key_version: "encryptionkey@0"
I20260925 10:54:21.528787 1332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260925 10:54:21.532557 1347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:21.534045 1332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:54:21.534235 1332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "eaefc446ad8a420cbb49ffda1e09a0be"
format_stamp: "Formatted at 2026-09-25 10:54:21 on dist-test-slave-nx7s"
server_key: "b0d550783cf71404effa80f656567ee8"
server_key_iv: "cc29d1474c4f41b6d159845fe018d8df"
server_key_version: "encryptionkey@0"
I20260925 10:54:21.534436 1332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:21.561194 1332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:21.565546 1332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:21.565807 1332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:21.568452 1332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:21.568559 1332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:21.568668 1332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:21.568745 1332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 10:54:21 dist-test-slave-nx7s krb5kdc[1216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333661, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:21.593609 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:21.569259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:42323 (local address 127.31.62.190:44035)
0925 10:54:21.569718 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:21.569724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:21.570978 (+ 1254us) server_negotiation.cc:407] Connection header received
0925 10:54:21.573283 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:21.573291 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:21.573398 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:21.573528 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:21.575914 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:21.576954 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:21.579717 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:21.580506 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:21.580658 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:21.585838 (+ 5180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:21.585846 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:21.585851 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:21.585899 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:21.589200 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:21.590459 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:21.590465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:21.590468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:21.590655 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:21.591263 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:21.591268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:21.591271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:21.591672 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:21.591824 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:21.592965 (+ 1141us) server_negotiation.cc:299] Negotiation successful
0925 10:54:21.593224 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":234,"threads_started":1}
I20260925 10:54:21.595393 1357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:21.569928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44035 (local address 127.31.62.129:42323)
0925 10:54:21.570782 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:21.570832 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:21.572802 (+ 1970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:21.573849 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:21.573868 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:21.574708 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:21.575689 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:21.575715 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:21.577160 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:21.577165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:21.579496 (+ 2331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:21.579518 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:21.580892 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:21.580903 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:21.581300 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:21.582228 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:21.582265 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:21.585547 (+ 3282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:21.589394 (+ 3847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:21.589410 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:21.589439 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:21.590263 (+ 824us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:21.590784 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:21.590790 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:21.590794 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:21.591113 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:21.591828 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:21.591842 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:21.592432 (+ 590us) client_negotiation.cc:769] Sending connection context
0925 10:54:21.592826 (+ 394us) client_negotiation.cc:240] Negotiation successful
0925 10:54:21.593559 (+ 733us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":358,"threads_started":1}
I20260925 10:54:21.606789 1332 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:42173
I20260925 10:54:21.606837 1467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:42173 every 8 connection(s)
I20260925 10:54:21.608163 1332 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:54:21.610615 1468 heartbeater.cc:347] Connected to a master server at 127.31.62.190:44035
I20260925 10:54:21.610846 1468 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:21.611519 1468 heartbeater.cc:511] Master 127.31.62.190:44035 requested a full tablet report, sending...
I20260925 10:54:21.613479 1261 ts_manager.cc:194] Registered new tserver with Master: eaefc446ad8a420cbb49ffda1e09a0be (127.31.62.129:42173)
I20260925 10:54:21.615280 1261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:42323
I20260925 10:54:21.615890 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 1332
I20260925 10:54:21.616070 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:54:21.616227 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:21.603427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:59175 (local address 127.31.62.190:44035)
0925 10:54:21.603644 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:21.603651 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:21.604678 (+ 1027us) server_negotiation.cc:407] Connection header received
0925 10:54:21.604908 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:21.604915 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:21.605005 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:21.605135 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:21.606035 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:21.607066 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:21.608317 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:21.608953 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:21.609674 (+ 721us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:21.611002 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:21.611007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:21.611010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:21.611038 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:21.613457 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:21.614164 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:21.614169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:21.614171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:21.614366 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:21.614753 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:21.614757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:21.614759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:21.615023 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:21.615120 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:21.615663 (+ 543us) server_negotiation.cc:299] Negotiation successful
0925 10:54:21.615894 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 10:54:21.616353 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:21.603699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44035 (local address 127.31.62.129:59175)
0925 10:54:21.604559 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:21.604578 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:21.604714 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:21.605157 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:21.605161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:21.605622 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:21.605900 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:21.605910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:21.607169 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:21.607172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:21.608179 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:21.608190 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:21.609178 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:21.609185 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:21.609330 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:21.610023 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:21.610046 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:21.610846 (+ 800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:21.613582 (+ 2736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:21.613587 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:21.613592 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:21.614053 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:21.614487 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:21.614492 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:21.614496 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:21.614651 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:21.615131 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:21.615136 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:21.615312 (+ 176us) client_negotiation.cc:769] Sending connection context
0925 10:54:21.615482 (+ 170us) client_negotiation.cc:240] Negotiation successful
0925 10:54:21.615791 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":584,"threads_started":1}
I20260925 10:54:21.616894 31994 external_mini_cluster.cc:1428] Setting key 9aff7a5216dd3e2ec5d0aadc7c7c54c2
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:54:21.656358 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:44035
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:21.971534 1472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:21.971818 1472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:21.971947 1472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:21.979301 1472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:21.979460 1472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:21.979589 1472 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 10:54:21.979652 1472 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 10:54:21.979720 1472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:54:21.989574 1472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.62.190:44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:21.990833 1472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:21.992205 1472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:22.005092 1480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:22.005124 1477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:22.005059 1478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:22.006067 1472 server_base.cc:1034] running on GCE node
I20260925 10:54:22.006762 1472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:22.007725 1472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:22.008961 1472 hybrid_clock.cc:648] HybridClock initialized: now 1790333662008901 us; error 47 us; skew 500 ppm
I20260925 10:54:22.009312 1472 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:22 dist-test-slave-nx7s krb5kdc[1216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333662, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:22.020265 1472 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:54:22.022120 1472 webserver.cc:460] Webserver started at http://127.31.62.130:43859/ using document root <none> and password file <none>
I20260925 10:54:22.023048 1472 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:22.023190 1472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:22.023542 1472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:22.027556 1472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "73b2e052b43c450e90ad693b70710718"
format_stamp: "Formatted at 2026-09-25 10:54:22 on dist-test-slave-nx7s"
server_key: "4882cf100435aa7168eca23966430b9c"
server_key_iv: "df6d8fec4cfe70bd232ba534289cb694"
server_key_version: "encryptionkey@0"
I20260925 10:54:22.028426 1472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "73b2e052b43c450e90ad693b70710718"
format_stamp: "Formatted at 2026-09-25 10:54:22 on dist-test-slave-nx7s"
server_key: "4882cf100435aa7168eca23966430b9c"
server_key_iv: "df6d8fec4cfe70bd232ba534289cb694"
server_key_version: "encryptionkey@0"
I20260925 10:54:22.033586 1472 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 10:54:22.037231 1487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:22.038733 1472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 10:54:22.038879 1472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "73b2e052b43c450e90ad693b70710718"
format_stamp: "Formatted at 2026-09-25 10:54:22 on dist-test-slave-nx7s"
server_key: "4882cf100435aa7168eca23966430b9c"
server_key_iv: "df6d8fec4cfe70bd232ba534289cb694"
server_key_version: "encryptionkey@0"
I20260925 10:54:22.039093 1472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:22.061717 1472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:22.066172 1472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:22.066419 1472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:22.069200 1472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:22.069334 1472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:22.069471 1472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:22.069607 1472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Sep 25 10:54:22 dist-test-slave-nx7s krb5kdc[1216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333662, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:22.092449 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.069637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:45775 (local address 127.31.62.190:44035)
0925 10:54:22.069820 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:22.069827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:22.071506 (+ 1679us) server_negotiation.cc:407] Connection header received
0925 10:54:22.073749 (+ 2243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:22.073755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:22.073840 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:22.073929 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:22.076306 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.077244 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.080549 (+ 3305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.081194 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.081341 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.086050 (+ 4709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:22.086056 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:22.086060 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:22.086096 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:22.089202 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:22.089978 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:22.089985 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:22.089989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:22.090111 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:22.090636 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:22.090640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:22.090642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:22.090939 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:22.091064 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:22.091887 (+ 823us) server_negotiation.cc:299] Negotiation successful
0925 10:54:22.092103 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 10:54:22.093856 1496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.070498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44035 (local address 127.31.62.130:45775)
0925 10:54:22.071327 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:22.071370 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:22.073392 (+ 2022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:22.074256 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:22.074273 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:22.075124 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:22.076084 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.076109 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.077473 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.077481 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.080306 (+ 2825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.080338 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.081488 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.081497 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.081772 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.082603 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:22.082659 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:22.085692 (+ 3033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:22.089362 (+ 3670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:22.089373 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:22.089393 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:22.089844 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:22.090229 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:22.090234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:22.090238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:22.090497 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:22.091071 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:22.091081 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:22.091506 (+ 425us) client_negotiation.cc:769] Sending connection context
0925 10:54:22.091797 (+ 291us) client_negotiation.cc:240] Negotiation successful
0925 10:54:22.092342 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":355,"threads_started":1}
I20260925 10:54:22.116772 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.100673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:41731 (local address 127.31.62.190:44035)
0925 10:54:22.100838 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:22.100843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:22.101582 (+ 739us) server_negotiation.cc:407] Connection header received
0925 10:54:22.101759 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:22.101764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:22.101845 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:22.101945 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:22.103401 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.104145 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.106316 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.106837 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.107007 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.109973 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:22.109981 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:22.109987 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:22.110037 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:22.113684 (+ 3647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:22.114519 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:22.114526 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:22.114529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:22.114726 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:22.115222 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:22.115226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:22.115228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:22.115665 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:22.115801 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:22.116198 (+ 397us) server_negotiation.cc:299] Negotiation successful
0925 10:54:22.116409 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 10:54:22.116901 1567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.100741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44035 (local address 127.31.62.130:41731)
0925 10:54:22.101470 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:22.101487 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:22.101640 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:22.102017 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:22.102025 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:22.102703 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:22.103162 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.103179 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.104341 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.104346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.106108 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.106129 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.107273 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.107280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.107475 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.108258 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:22.108279 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:22.109679 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:22.113880 (+ 4201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:22.113884 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:22.113887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:22.114362 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:22.114907 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:22.114912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:22.114915 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:22.115093 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:22.115777 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:22.115782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:22.115958 (+ 176us) client_negotiation.cc:769] Sending connection context
0925 10:54:22.116067 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 10:54:22.116396 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":403,"threads_started":1}
I20260925 10:54:22.128955 1472 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:34269
I20260925 10:54:22.129004 1606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:34269 every 8 connection(s)
I20260925 10:54:22.130643 1472 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:54:22.133616 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 1472
I20260925 10:54:22.133795 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:54:22.134501 31994 external_mini_cluster.cc:1428] Setting key 62a8e53a2e1f805b42c688134c6921b6
I20260925 10:54:22.135164 1607 heartbeater.cc:347] Connected to a master server at 127.31.62.190:44035
I20260925 10:54:22.135574 1607 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:22.137313 1607 heartbeater.cc:511] Master 127.31.62.190:44035 requested a full tablet report, sending...
I20260925 10:54:22.139012 1261 ts_manager.cc:194] Registered new tserver with Master: 73b2e052b43c450e90ad693b70710718 (127.31.62.130:34269)
I20260925 10:54:22.140141 1261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:45775
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:54:22.179428 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:44035
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:22.499092 1611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:22.499393 1611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:22.499537 1611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:22.506804 1611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:22.506927 1611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:22.507052 1611 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 10:54:22.507118 1611 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 10:54:22.507221 1611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:54:22.517242 1611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.62.190:44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:22.518507 1611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:22.519763 1611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:22.532415 1617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:22.532485 1619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:22.532449 1616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:22.533574 1611 server_base.cc:1034] running on GCE node
I20260925 10:54:22.534227 1611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:22.535261 1611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:22.536494 1611 hybrid_clock.cc:648] HybridClock initialized: now 1790333662536425 us; error 52 us; skew 500 ppm
I20260925 10:54:22.536873 1611 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:22 dist-test-slave-nx7s krb5kdc[1216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333662, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:22.540189 1611 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:54:22.541745 1611 webserver.cc:460] Webserver started at http://127.31.62.131:37367/ using document root <none> and password file <none>
I20260925 10:54:22.542501 1611 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:22.542623 1611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:22.542918 1611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:22.545855 1611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "a56110511b1f4138a7fb8255b33def19"
format_stamp: "Formatted at 2026-09-25 10:54:22 on dist-test-slave-nx7s"
server_key: "85a31788938478616f3d707fbdb19258"
server_key_iv: "b6b36c2cb3ea964e6b08dd70c46a169d"
server_key_version: "encryptionkey@0"
I20260925 10:54:22.546695 1611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "a56110511b1f4138a7fb8255b33def19"
format_stamp: "Formatted at 2026-09-25 10:54:22 on dist-test-slave-nx7s"
server_key: "85a31788938478616f3d707fbdb19258"
server_key_iv: "b6b36c2cb3ea964e6b08dd70c46a169d"
server_key_version: "encryptionkey@0"
I20260925 10:54:22.551988 1611 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:54:22.555956 1626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:22.557561 1611 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:54:22.557723 1611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "a56110511b1f4138a7fb8255b33def19"
format_stamp: "Formatted at 2026-09-25 10:54:22 on dist-test-slave-nx7s"
server_key: "85a31788938478616f3d707fbdb19258"
server_key_iv: "b6b36c2cb3ea964e6b08dd70c46a169d"
server_key_version: "encryptionkey@0"
I20260925 10:54:22.557931 1611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:22.578267 1611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:22.582729 1611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:22.583001 1611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:22.585916 1611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:22.586033 1611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:22.586144 1611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:22.586218 1611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 10:54:22 dist-test-slave-nx7s krb5kdc[1216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333662, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:22.611966 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.586564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:40131 (local address 127.31.62.190:44035)
0925 10:54:22.586923 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:22.586929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:22.588329 (+ 1400us) server_negotiation.cc:407] Connection header received
0925 10:54:22.590856 (+ 2527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:22.590861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:22.590950 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:22.591043 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:22.593735 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.594703 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.597430 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.598009 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.598138 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.603262 (+ 5124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:22.603267 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:22.603271 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:22.603308 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:22.606657 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:22.607796 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:22.607801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:22.607805 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:22.607973 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:22.608593 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:22.608598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:22.608602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:22.608990 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:22.609126 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:22.611503 (+ 2377us) server_negotiation.cc:299] Negotiation successful
0925 10:54:22.611728 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234}
I20260925 10:54:22.612327 1635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.587257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44035 (local address 127.31.62.131:40131)
0925 10:54:22.588123 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:22.588193 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:22.590514 (+ 2321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:22.591380 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:22.591394 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:22.592315 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:22.593460 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.593493 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.594917 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.594924 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.597242 (+ 2318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.597261 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.598258 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.598265 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.598601 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.599439 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:22.599488 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:22.602892 (+ 3404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:22.606837 (+ 3945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:22.606849 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:22.606871 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:22.607631 (+ 760us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:22.608111 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:22.608116 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:22.608120 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:22.608431 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:22.609216 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:22.609223 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:22.609718 (+ 495us) client_negotiation.cc:769] Sending connection context
0925 10:54:22.610030 (+ 312us) client_negotiation.cc:240] Negotiation successful
0925 10:54:22.610665 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":360,"threads_started":1}
I20260925 10:54:22.619202 1468 heartbeater.cc:503] Master 127.31.62.190:44035 was elected leader, sending a full tablet report...
I20260925 10:54:22.628651 1611 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:33723
I20260925 10:54:22.628651 1745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:33723 every 8 connection(s)
I20260925 10:54:22.630057 1611 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:54:22.632231 1746 heartbeater.cc:347] Connected to a master server at 127.31.62.190:44035
I20260925 10:54:22.632486 1746 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:22.633460 1746 heartbeater.cc:511] Master 127.31.62.190:44035 requested a full tablet report, sending...
I20260925 10:54:22.634805 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.619485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:58257 (local address 127.31.62.190:44035)
0925 10:54:22.619657 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:22.619663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:22.620230 (+ 567us) server_negotiation.cc:407] Connection header received
0925 10:54:22.620477 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:22.620483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:22.620594 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:22.620699 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:22.622308 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.623007 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.625403 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.626066 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.626190 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.628791 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:22.628798 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:22.628803 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:22.628867 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:22.631676 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:22.632503 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:22.632509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:22.632513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:22.632667 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:22.633133 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:22.633138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:22.633140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:22.633694 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:22.633830 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:22.634246 (+ 416us) server_negotiation.cc:299] Negotiation successful
0925 10:54:22.634448 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 10:54:22.634886 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.619532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44035 (local address 127.31.62.131:58257)
0925 10:54:22.620131 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:22.620150 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:22.620345 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:22.620919 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:22.620927 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:22.621651 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:22.622153 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.622167 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.623158 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.623164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.625171 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.625191 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.626530 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.626536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.626730 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.627346 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:22.627368 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:22.628530 (+ 1162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:22.631829 (+ 3299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:22.631834 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:22.631839 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:22.632344 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:22.632797 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:22.632802 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:22.632805 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:22.632986 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:22.633805 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:22.633810 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:22.633997 (+ 187us) client_negotiation.cc:769] Sending connection context
0925 10:54:22.634139 (+ 142us) client_negotiation.cc:240] Negotiation successful
0925 10:54:22.634454 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":430,"threads_started":1}
I20260925 10:54:22.635437 1261 ts_manager.cc:194] Registered new tserver with Master: a56110511b1f4138a7fb8255b33def19 (127.31.62.131:33723)
I20260925 10:54:22.636529 1261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:40131
I20260925 10:54:22.636931 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 1611
I20260925 10:54:22.637104 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:54:22.637661 31994 external_mini_cluster.cc:1428] Setting key af893da2b9ae524b45175a55979bb872
I20260925 10:54:22.640496 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 10:54:22 dist-test-slave-nx7s krb5kdc[1216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 10:54:22 dist-test-slave-nx7s krb5kdc[1216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:22.683929 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.671100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55266 (local address 127.31.62.190:44035)
0925 10:54:22.671377 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:22.671383 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:22.671482 (+ 99us) server_negotiation.cc:407] Connection header received
0925 10:54:22.671664 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:22.671668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:22.671753 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:22.671838 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:22.673218 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.673948 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.675173 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.675592 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.675704 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.679425 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:22.679430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:22.679433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:22.679468 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:22.681775 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:22.682345 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:22.682348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:22.682350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:22.682456 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:22.682749 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:22.682752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:22.682754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:22.683019 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:22.683178 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:22.683438 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 10:54:22.683592 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260925 10:54:22.687809 1261 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:54:22.694538 1261 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:54:22.715935 1761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.706667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48746 (local address 127.31.62.130:34269)
0925 10:54:22.708151 (+ 1484us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:22.708159 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:22.708187 (+ 28us) server_negotiation.cc:407] Connection header received
0925 10:54:22.708280 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:22.708289 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:22.708569 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:22.709560 (+ 991us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:22.709707 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.710588 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.713228 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.714566 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.714706 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.714905 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:22.715403 (+ 498us) server_negotiation.cc:299] Negotiation successful
0925 10:54:22.715580 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1336,"thread_start_us":1232,"threads_started":1}
I20260925 10:54:22.717008 1759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.706477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:34269 (local address 127.0.0.1:48746)
0925 10:54:22.707365 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:22.707418 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:22.707612 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:22.708802 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:22.708810 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:22.708882 (+ 72us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:22.709346 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.709358 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.710720 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.710725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.713034 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.713051 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.714787 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.714794 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.715023 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.715184 (+ 161us) client_negotiation.cc:769] Sending connection context
0925 10:54:22.715342 (+ 158us) client_negotiation.cc:240] Negotiation successful
0925 10:54:22.716563 (+ 1221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":172,"thread_start_us":893,"threads_started":1}
I20260925 10:54:22.719296 1764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.709198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52288 (local address 127.31.62.129:42173)
0925 10:54:22.711520 (+ 2322us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:22.711530 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:22.712501 (+ 971us) server_negotiation.cc:407] Connection header received
0925 10:54:22.712640 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:22.712647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:22.712900 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:22.713073 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:22.713645 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.714630 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.717039 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.718376 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.718483 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.718680 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:22.718876 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 10:54:22.719027 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2180,"thread_start_us":2055,"threads_started":1}
I20260925 10:54:22.719291 1760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.708687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:42173 (local address 127.0.0.1:52288)
0925 10:54:22.712376 (+ 3689us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:22.712394 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:22.712507 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:22.713191 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:22.713195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:22.713217 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:22.713499 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.713507 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.714810 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.714814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.716890 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.716907 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.718551 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.718556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.718717 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.718748 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 10:54:22.718827 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 10:54:22.718943 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3574,"mutex_wait_us":1,"thread_start_us":675,"threads_started":1}
I20260925 10:54:22.720588 1763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.706596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:33723 (local address 127.0.0.1:41768)
0925 10:54:22.709275 (+ 2679us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:22.709297 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:22.709406 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:22.712648 (+ 3242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:22.712653 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:22.712674 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:22.713087 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.713097 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.714999 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.715004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.718121 (+ 3117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.718136 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.719827 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.719834 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.719992 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.720023 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 10:54:22.720092 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 10:54:22.720220 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2559,"mutex_wait_us":42,"thread_start_us":5546,"threads_started":1}
I20260925 10:54:22.721346 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.707153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41768 (local address 127.31.62.131:33723)
0925 10:54:22.711204 (+ 4051us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:22.711212 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:22.711243 (+ 31us) server_negotiation.cc:407] Connection header received
0925 10:54:22.711339 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:22.711346 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:22.711592 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:22.711769 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:22.713270 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.714889 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.718274 (+ 3385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.719646 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.719771 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.719969 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:22.720896 (+ 927us) server_negotiation.cc:299] Negotiation successful
0925 10:54:22.721056 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3883,"thread_start_us":3708,"threads_started":1}
I20260925 10:54:22.720933 1537 tablet_service.cc:1467] Processing CreateTablet for tablet 15529422f8ef4b4a94838757c49d9df0 (DEFAULT_TABLE table=test-table [id=5a30a8ea6e8b48588dc5444beb1c5b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:22.722417 1537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15529422f8ef4b4a94838757c49d9df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:22.724466 1398 tablet_service.cc:1467] Processing CreateTablet for tablet 15529422f8ef4b4a94838757c49d9df0 (DEFAULT_TABLE table=test-table [id=5a30a8ea6e8b48588dc5444beb1c5b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:22.725157 1676 tablet_service.cc:1467] Processing CreateTablet for tablet 15529422f8ef4b4a94838757c49d9df0 (DEFAULT_TABLE table=test-table [id=5a30a8ea6e8b48588dc5444beb1c5b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:22.725996 1398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15529422f8ef4b4a94838757c49d9df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:22.726634 1676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15529422f8ef4b4a94838757c49d9df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:22.733840 1765 tablet_bootstrap.cc:492] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718: Bootstrap starting.
I20260925 10:54:22.736801 1765 tablet_bootstrap.cc:654] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:22.737823 1765 log.cc:824] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:22.740281 1767 tablet_bootstrap.cc:492] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be: Bootstrap starting.
I20260925 10:54:22.744644 1767 tablet_bootstrap.cc:654] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:22.745671 1768 tablet_bootstrap.cc:492] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19: Bootstrap starting.
I20260925 10:54:22.745990 1767 log.cc:824] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:22.747777 1765 tablet_bootstrap.cc:492] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718: No bootstrap required, opened a new log
I20260925 10:54:22.748054 1765 ts_tablet_manager.cc:1397] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260925 10:54:22.748948 1767 tablet_bootstrap.cc:492] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be: No bootstrap required, opened a new log
I20260925 10:54:22.749200 1767 ts_tablet_manager.cc:1397] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260925 10:54:22.751642 1768 tablet_bootstrap.cc:654] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:22.752624 1768 log.cc:824] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:22.756017 1768 tablet_bootstrap.cc:492] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19: No bootstrap required, opened a new log
I20260925 10:54:22.756266 1768 ts_tablet_manager.cc:1397] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260925 10:54:22.758270 1765 raft_consensus.cc:348] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } }
I20260925 10:54:22.758854 1765 raft_consensus.cc:374] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:22.758993 1765 raft_consensus.cc:729] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73b2e052b43c450e90ad693b70710718, State: Initialized, Role: FOLLOWER
I20260925 10:54:22.759150 1767 raft_consensus.cc:348] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } }
I20260925 10:54:22.759546 1767 raft_consensus.cc:374] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:22.759639 1767 raft_consensus.cc:729] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaefc446ad8a420cbb49ffda1e09a0be, State: Initialized, Role: FOLLOWER
I20260925 10:54:22.759621 1765 consensus_queue.cc:260] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } }
I20260925 10:54:22.760193 1767 consensus_queue.cc:260] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } }
I20260925 10:54:22.760748 1607 heartbeater.cc:503] Master 127.31.62.190:44035 was elected leader, sending a full tablet report...
I20260925 10:54:22.761494 1765 ts_tablet_manager.cc:1428] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260925 10:54:22.764123 1767 ts_tablet_manager.cc:1428] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be: Time spent starting tablet: real 0.015s user 0.008s sys 0.004s
I20260925 10:54:22.764986 1768 raft_consensus.cc:348] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } }
I20260925 10:54:22.765430 1768 raft_consensus.cc:374] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:22.765534 1768 raft_consensus.cc:729] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a56110511b1f4138a7fb8255b33def19, State: Initialized, Role: FOLLOWER
I20260925 10:54:22.766148 1768 consensus_queue.cc:260] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } }
I20260925 10:54:22.767330 1746 heartbeater.cc:503] Master 127.31.62.190:44035 was elected leader, sending a full tablet report...
I20260925 10:54:22.767843 1768 ts_tablet_manager.cc:1428] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19: Time spent starting tablet: real 0.011s user 0.008s sys 0.005s
W20260925 10:54:22.860539 1469 tablet.cc:2369] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:54:22.870011 1772 raft_consensus.cc:482] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:54:22.870313 1772 raft_consensus.cc:504] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } }
I20260925 10:54:22.871970 1772 leader_election.cc:288] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73b2e052b43c450e90ad693b70710718 (127.31.62.130:34269), a56110511b1f4138a7fb8255b33def19 (127.31.62.131:33723)
I20260925 10:54:22.880784 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.872384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:44833 (local address 127.31.62.131:33723)
0925 10:54:22.872648 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:22.872657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:22.873346 (+ 689us) server_negotiation.cc:407] Connection header received
0925 10:54:22.873419 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:22.873424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:22.873488 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:22.873568 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:22.874217 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.875317 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.878345 (+ 3028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.879626 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.879828 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.879907 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:22.880374 (+ 467us) server_negotiation.cc:299] Negotiation successful
0925 10:54:22.880500 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260925 10:54:22.880795 1761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.872389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:36767 (local address 127.31.62.130:34269)
0925 10:54:22.872829 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:22.872837 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:22.873084 (+ 247us) server_negotiation.cc:407] Connection header received
0925 10:54:22.873319 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:22.873325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:22.873440 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:22.873547 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:22.874286 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.875317 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.878346 (+ 3029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.879676 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.879862 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.879926 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:22.880365 (+ 439us) server_negotiation.cc:299] Negotiation successful
0925 10:54:22.880500 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299}
I20260925 10:54:22.881101 1775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.872264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:34269 (local address 127.31.62.129:36767)
0925 10:54:22.872946 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:22.872967 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:22.873142 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:22.873569 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:22.873574 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:22.873597 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:22.874077 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.874086 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.875469 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.875475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.878092 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.878115 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.879901 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.879906 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.880174 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.880219 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 10:54:22.880327 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 10:54:22.880556 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":240,"thread_start_us":687,"threads_started":1}
I20260925 10:54:22.881121 1774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.872277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:33723 (local address 127.31.62.129:44833)
0925 10:54:22.873224 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:22.873239 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:22.873341 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:22.873575 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:22.873579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:22.873626 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:22.874065 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.874078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.875470 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.875475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.878092 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.878115 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.879888 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.879895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.880159 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.880219 (+ 60us) client_negotiation.cc:769] Sending connection context
0925 10:54:22.880325 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 10:54:22.880556 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":831,"mutex_wait_us":222,"spinlock_wait_cycles":28544,"thread_start_us":297,"threads_started":1}
I20260925 10:54:22.882233 1696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15529422f8ef4b4a94838757c49d9df0" candidate_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a56110511b1f4138a7fb8255b33def19" is_pre_election: true
I20260925 10:54:22.882243 1557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15529422f8ef4b4a94838757c49d9df0" candidate_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b2e052b43c450e90ad693b70710718" is_pre_election: true
I20260925 10:54:22.882812 1557 raft_consensus.cc:2454] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaefc446ad8a420cbb49ffda1e09a0be in term 0.
I20260925 10:54:22.882812 1696 raft_consensus.cc:2454] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaefc446ad8a420cbb49ffda1e09a0be in term 0.
I20260925 10:54:22.883595 1350 leader_election.cc:302] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a56110511b1f4138a7fb8255b33def19, eaefc446ad8a420cbb49ffda1e09a0be; no voters:
W20260925 10:54:22.883729 1608 tablet.cc:2369] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:54:22.883729 1747 tablet.cc:2369] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:54:22.884078 1772 raft_consensus.cc:2781] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:54:22.884235 1772 raft_consensus.cc:482] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:54:22.884322 1772 raft_consensus.cc:3037] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:22.886502 1772 raft_consensus.cc:504] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } }
I20260925 10:54:22.887115 1772 leader_election.cc:288] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [CANDIDATE]: Term 1 election: Requested vote from peers 73b2e052b43c450e90ad693b70710718 (127.31.62.130:34269), a56110511b1f4138a7fb8255b33def19 (127.31.62.131:33723)
I20260925 10:54:22.887614 1557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15529422f8ef4b4a94838757c49d9df0" candidate_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b2e052b43c450e90ad693b70710718"
I20260925 10:54:22.887620 1696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15529422f8ef4b4a94838757c49d9df0" candidate_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a56110511b1f4138a7fb8255b33def19"
I20260925 10:54:22.887797 1557 raft_consensus.cc:3037] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:22.887807 1696 raft_consensus.cc:3037] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:22.889990 1557 raft_consensus.cc:2454] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaefc446ad8a420cbb49ffda1e09a0be in term 1.
I20260925 10:54:22.889990 1696 raft_consensus.cc:2454] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaefc446ad8a420cbb49ffda1e09a0be in term 1.
I20260925 10:54:22.890416 1350 leader_election.cc:302] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a56110511b1f4138a7fb8255b33def19, eaefc446ad8a420cbb49ffda1e09a0be; no voters:
I20260925 10:54:22.890717 1772 raft_consensus.cc:2781] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:22.891124 1772 raft_consensus.cc:686] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [term 1 LEADER]: Becoming Leader. State: Replica: eaefc446ad8a420cbb49ffda1e09a0be, State: Running, Role: LEADER
I20260925 10:54:22.891582 1772 consensus_queue.cc:237] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } }
I20260925 10:54:22.896471 1261 catalog_manager.cc:5495] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be reported cstate change: term changed from 0 to 1, leader changed from <none> to eaefc446ad8a420cbb49ffda1e09a0be (127.31.62.129). New cstate: current_term: 1 leader_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } health_report { overall_health: HEALTHY } } }
Sep 25 10:54:22 dist-test-slave-nx7s krb5kdc[1216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333660, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:22.991336 1783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.971291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44035 (local address 127.0.0.1:55272)
0925 10:54:22.971862 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:22.971878 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:22.972004 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:22.972380 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:22.972384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:22.973139 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:22.973427 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.973438 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.974480 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.974484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.975440 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:22.975451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.976171 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.976175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:22.976301 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.983660 (+ 7359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:22.983682 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:22.986070 (+ 2388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:22.989046 (+ 2976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:22.989059 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:22.989066 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:22.989480 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:22.989836 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:22.989839 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:22.989841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:22.989980 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:22.990446 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:22.990452 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:22.990610 (+ 158us) client_negotiation.cc:769] Sending connection context
0925 10:54:22.990738 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 10:54:22.990996 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":283,"threads_started":1}
I20260925 10:54:22.991395 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:22.971292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55272 (local address 127.31.62.190:44035)
0925 10:54:22.971454 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:22.971458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:22.971979 (+ 521us) server_negotiation.cc:407] Connection header received
0925 10:54:22.972103 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:22.972151 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:22.972242 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:22.972341 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:22.973577 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.974361 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.975584 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:22.976019 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:22.976110 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:22.986288 (+ 10178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:22.986294 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:22.986298 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:22.986334 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:22.988905 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:22.989595 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:22.989599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:22.989601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:22.989727 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:22.990061 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:22.990064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:22.990066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:22.990350 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:22.990458 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:22.990852 (+ 394us) server_negotiation.cc:299] Negotiation successful
0925 10:54:22.991102 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":53}
I20260925 10:54:22.996418 1261 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:55272:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 10:54:22.997466 1261 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:54:23.004132 1537 tablet_service.cc:1467] Processing CreateTablet for tablet 820f6bd4f0434faca1631e8a79b947a9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bc69a2a9f6bb491fadeba9cf7c5e3cb0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 10:54:23.003886 1398 tablet_service.cc:1467] Processing CreateTablet for tablet 820f6bd4f0434faca1631e8a79b947a9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bc69a2a9f6bb491fadeba9cf7c5e3cb0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 10:54:23.004184 1676 tablet_service.cc:1467] Processing CreateTablet for tablet 820f6bd4f0434faca1631e8a79b947a9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bc69a2a9f6bb491fadeba9cf7c5e3cb0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 10:54:23.004956 1398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 820f6bd4f0434faca1631e8a79b947a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:23.004956 1537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 820f6bd4f0434faca1631e8a79b947a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:23.004972 1676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 820f6bd4f0434faca1631e8a79b947a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:23.009837 1768 tablet_bootstrap.cc:492] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19: Bootstrap starting.
I20260925 10:54:23.011399 1768 tablet_bootstrap.cc:654] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:23.011744 1767 tablet_bootstrap.cc:492] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be: Bootstrap starting.
I20260925 10:54:23.011744 1765 tablet_bootstrap.cc:492] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718: Bootstrap starting.
I20260925 10:54:23.013263 1765 tablet_bootstrap.cc:654] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:23.013263 1767 tablet_bootstrap.cc:654] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:23.022585 1765 tablet_bootstrap.cc:492] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718: No bootstrap required, opened a new log
I20260925 10:54:23.022573 1767 tablet_bootstrap.cc:492] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be: No bootstrap required, opened a new log
I20260925 10:54:23.022730 1767 ts_tablet_manager.cc:1397] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.002s
I20260925 10:54:23.022734 1765 ts_tablet_manager.cc:1397] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260925 10:54:23.023082 1768 tablet_bootstrap.cc:492] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19: No bootstrap required, opened a new log
I20260925 10:54:23.023218 1768 ts_tablet_manager.cc:1397] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260925 10:54:23.023512 1765 raft_consensus.cc:348] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.023499 1767 raft_consensus.cc:348] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.023730 1765 raft_consensus.cc:374] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:23.023741 1767 raft_consensus.cc:374] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:23.023819 1767 raft_consensus.cc:729] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaefc446ad8a420cbb49ffda1e09a0be, State: Initialized, Role: FOLLOWER
I20260925 10:54:23.023824 1765 raft_consensus.cc:729] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73b2e052b43c450e90ad693b70710718, State: Initialized, Role: FOLLOWER
I20260925 10:54:23.023985 1768 raft_consensus.cc:348] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.024026 1767 consensus_queue.cc:260] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.024212 1768 raft_consensus.cc:374] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:23.024106 1765 consensus_queue.cc:260] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.024299 1768 raft_consensus.cc:729] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a56110511b1f4138a7fb8255b33def19, State: Initialized, Role: FOLLOWER
I20260925 10:54:23.024552 1767 ts_tablet_manager.cc:1428] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260925 10:54:23.024525 1771 tablet_replica.cc:439] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } }
I20260925 10:54:23.024570 1768 consensus_queue.cc:260] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.024983 1772 tablet_replica.cc:439] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } }
I20260925 10:54:23.025182 1768 ts_tablet_manager.cc:1428] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 10:54:23.025241 1772 tablet_replica.cc:442] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:54:23.025959 1773 tablet_replica.cc:439] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } }
I20260925 10:54:23.026309 1773 tablet_replica.cc:442] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:54:23.027282 1771 tablet_replica.cc:442] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:54:23.028067 1765 ts_tablet_manager.cc:1428] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260925 10:54:23.145125 1773 raft_consensus.cc:482] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:54:23.145308 1773 raft_consensus.cc:504] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.146657 1773 leader_election.cc:288] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eaefc446ad8a420cbb49ffda1e09a0be (127.31.62.129:42173), 73b2e052b43c450e90ad693b70710718 (127.31.62.130:34269)
I20260925 10:54:23.153765 1761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:23.147114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:55437 (local address 127.31.62.130:34269)
0925 10:54:23.147277 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:23.147283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:23.147745 (+ 462us) server_negotiation.cc:407] Connection header received
0925 10:54:23.147945 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:23.147951 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:23.148028 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:23.148194 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:23.148719 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.149911 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.152026 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.152804 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.152980 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:23.153072 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:23.153353 (+ 281us) server_negotiation.cc:299] Negotiation successful
0925 10:54:23.153479 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 10:54:23.153875 1764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:23.147107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:36437 (local address 127.31.62.129:42173)
0925 10:54:23.147308 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:23.147315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:23.147893 (+ 578us) server_negotiation.cc:407] Connection header received
0925 10:54:23.148009 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:23.148014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:23.148093 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:23.148182 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:23.148665 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.149844 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.152029 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.152822 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.152980 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:23.153038 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:23.153493 (+ 455us) server_negotiation.cc:299] Negotiation successful
0925 10:54:23.153619 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 10:54:23.153960 1788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:23.146944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:34269 (local address 127.31.62.131:55437)
0925 10:54:23.147620 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:23.147640 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:23.147804 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:23.148199 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:23.148204 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:23.148299 (+ 95us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:23.148576 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:23.148585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.150039 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.150044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:23.151887 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:23.151900 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.152980 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.152987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:23.153174 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:23.153222 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 10:54:23.153328 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 10:54:23.153468 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":73,"spinlock_wait_cycles":21376,"thread_start_us":652,"threads_started":1}
I20260925 10:54:23.154170 1787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:23.146944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:42173 (local address 127.31.62.131:36437)
0925 10:54:23.147804 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:23.147818 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:23.147910 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:23.148199 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:23.148204 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:23.148228 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:23.148514 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:23.148529 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.150036 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.150040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:23.151883 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:23.151897 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.153171 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.153175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:23.153350 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:23.153373 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 10:54:23.153449 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 10:54:23.153574 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":126,"thread_start_us":408,"threads_started":1}
I20260925 10:54:23.154605 1557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820f6bd4f0434faca1631e8a79b947a9" candidate_uuid: "a56110511b1f4138a7fb8255b33def19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b2e052b43c450e90ad693b70710718" is_pre_election: true
I20260925 10:54:23.154834 1557 raft_consensus.cc:2454] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a56110511b1f4138a7fb8255b33def19 in term 0.
I20260925 10:54:23.155048 1418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820f6bd4f0434faca1631e8a79b947a9" candidate_uuid: "a56110511b1f4138a7fb8255b33def19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" is_pre_election: true
I20260925 10:54:23.155282 1630 leader_election.cc:302] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73b2e052b43c450e90ad693b70710718, a56110511b1f4138a7fb8255b33def19; no voters:
I20260925 10:54:23.155472 1418 raft_consensus.cc:2454] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a56110511b1f4138a7fb8255b33def19 in term 0.
I20260925 10:54:23.155727 1773 raft_consensus.cc:2781] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:54:23.155835 1773 raft_consensus.cc:482] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:54:23.155932 1773 raft_consensus.cc:3037] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:23.158128 1773 raft_consensus.cc:504] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.158659 1773 leader_election.cc:288] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [CANDIDATE]: Term 1 election: Requested vote from peers eaefc446ad8a420cbb49ffda1e09a0be (127.31.62.129:42173), 73b2e052b43c450e90ad693b70710718 (127.31.62.130:34269)
I20260925 10:54:23.159133 1557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820f6bd4f0434faca1631e8a79b947a9" candidate_uuid: "a56110511b1f4138a7fb8255b33def19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b2e052b43c450e90ad693b70710718"
I20260925 10:54:23.159152 1418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820f6bd4f0434faca1631e8a79b947a9" candidate_uuid: "a56110511b1f4138a7fb8255b33def19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefc446ad8a420cbb49ffda1e09a0be"
I20260925 10:54:23.159354 1557 raft_consensus.cc:3037] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:23.159373 1418 raft_consensus.cc:3037] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:23.162302 1557 raft_consensus.cc:2454] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a56110511b1f4138a7fb8255b33def19 in term 1.
I20260925 10:54:23.162326 1418 raft_consensus.cc:2454] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a56110511b1f4138a7fb8255b33def19 in term 1.
I20260925 10:54:23.162743 1628 leader_election.cc:302] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a56110511b1f4138a7fb8255b33def19, eaefc446ad8a420cbb49ffda1e09a0be; no voters:
I20260925 10:54:23.163095 1773 raft_consensus.cc:2781] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:23.163475 1773 raft_consensus.cc:686] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [term 1 LEADER]: Becoming Leader. State: Replica: a56110511b1f4138a7fb8255b33def19, State: Running, Role: LEADER
I20260925 10:54:23.164003 1773 consensus_queue.cc:237] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.167280 1789 tablet_replica.cc:439] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19: TxnStatusTablet state changed. Reason: New leader a56110511b1f4138a7fb8255b33def19. Latest consensus state: current_term: 1 leader_uuid: "a56110511b1f4138a7fb8255b33def19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } }
I20260925 10:54:23.167743 1789 tablet_replica.cc:442] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19: This TxnStatusTablet replica's current role is: LEADER
I20260925 10:54:23.168388 1791 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 10:54:23.168642 1791 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 10:54:23.168985 1260 catalog_manager.cc:5495] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 reported cstate change: term changed from 0 to 1, leader changed from <none> to a56110511b1f4138a7fb8255b33def19 (127.31.62.131). New cstate: current_term: 1 leader_uuid: "a56110511b1f4138a7fb8255b33def19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } health_report { overall_health: HEALTHY } } }
I20260925 10:54:23.195986 1783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:23.190144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:33723 (local address 127.0.0.1:41782)
0925 10:54:23.190425 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:23.190441 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:23.190591 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:23.191077 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:23.191082 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:23.191094 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 10:54:23.191270 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:23.191279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.192360 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.192366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:23.193622 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:23.193635 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.194379 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.194385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:23.194541 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:23.194600 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 10:54:23.195438 (+ 838us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 10:54:23.195462 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 10:54:23.195540 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 10:54:23.195692 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260925 10:54:23.196008 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:23.190172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41782 (local address 127.31.62.131:33723)
0925 10:54:23.190370 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:23.190376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:23.190508 (+ 132us) server_negotiation.cc:407] Connection header received
0925 10:54:23.190809 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:23.190813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:23.190916 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:23.191038 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:54:23.191393 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.192233 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.193786 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.194212 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.194333 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:23.194780 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:54:23.195333 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:54:23.195469 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:23.195573 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 10:54:23.195655 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 10:54:23.204562 1557 raft_consensus.cc:1261] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Refusing update from remote peer a56110511b1f4138a7fb8255b33def19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:23.204537 1418 raft_consensus.cc:1261] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Refusing update from remote peer a56110511b1f4138a7fb8255b33def19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:23.205673 1789 consensus_queue.cc:1035] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:23.206100 1773 consensus_queue.cc:1035] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:23.216418 1772 tablet_replica.cc:439] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be: TxnStatusTablet state changed. Reason: New leader a56110511b1f4138a7fb8255b33def19. Latest consensus state: current_term: 1 leader_uuid: "a56110511b1f4138a7fb8255b33def19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } }
I20260925 10:54:23.216872 1772 tablet_replica.cc:442] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:54:23.218097 1771 tablet_replica.cc:439] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718: TxnStatusTablet state changed. Reason: New leader a56110511b1f4138a7fb8255b33def19. Latest consensus state: current_term: 1 leader_uuid: "a56110511b1f4138a7fb8255b33def19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } }
I20260925 10:54:23.218396 1771 tablet_replica.cc:442] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:54:23.220898 1794 mvcc.cc:205] Tried to move back new op lower bound from 7333206684474834944 to 7333206684323577856. Current Snapshot: MvccSnapshot[applied={T|T < 7333206684474834944}]
I20260925 10:54:23.227080 1771 tablet_replica.cc:439] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a56110511b1f4138a7fb8255b33def19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } }
I20260925 10:54:23.227361 1771 tablet_replica.cc:442] T 820f6bd4f0434faca1631e8a79b947a9 P 73b2e052b43c450e90ad693b70710718: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:54:23.228068 1772 tablet_replica.cc:439] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a56110511b1f4138a7fb8255b33def19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } }
I20260925 10:54:23.228348 1772 tablet_replica.cc:442] T 820f6bd4f0434faca1631e8a79b947a9 P eaefc446ad8a420cbb49ffda1e09a0be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:54:23.232151 1789 tablet_replica.cc:439] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a56110511b1f4138a7fb8255b33def19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } }
I20260925 10:54:23.232460 1789 tablet_replica.cc:442] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19: This TxnStatusTablet replica's current role is: LEADER
I20260925 10:54:23.232923 1773 tablet_replica.cc:439] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a56110511b1f4138a7fb8255b33def19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } } }
I20260925 10:54:23.233189 1773 tablet_replica.cc:442] T 820f6bd4f0434faca1631e8a79b947a9 P a56110511b1f4138a7fb8255b33def19: This TxnStatusTablet replica's current role is: LEADER
I20260925 10:54:23.245566 1764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:23.239665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52302 (local address 127.31.62.129:42173)
0925 10:54:23.239829 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:23.239835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:23.240304 (+ 469us) server_negotiation.cc:407] Connection header received
0925 10:54:23.240527 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:23.240531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:23.240629 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:23.240762 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:54:23.241226 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.241957 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.243388 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.243784 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.243918 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:23.244330 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:54:23.244940 (+ 610us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:54:23.245075 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:23.245160 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 10:54:23.245246 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 10:54:23.262892 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:23.256657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:33025 (local address 127.31.62.131:33723)
0925 10:54:23.256822 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:23.256830 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:23.257423 (+ 593us) server_negotiation.cc:407] Connection header received
0925 10:54:23.257658 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:23.257662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:23.257738 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:23.257821 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:54:23.258366 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.259242 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.260787 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.261356 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.261505 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:23.262087 (+ 582us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:54:23.262226 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:54:23.262304 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:23.262494 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 10:54:23.262587 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 10:54:23.262982 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:23.256611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:33723 (local address 127.31.62.129:33025)
0925 10:54:23.257293 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:23.257310 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:23.257480 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:23.257866 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:23.257871 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:23.257884 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 10:54:23.258209 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:23.258219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.259398 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.259403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:23.260635 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:23.260650 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.261552 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.261559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:23.261751 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:23.261841 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 10:54:23.262325 (+ 484us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 10:54:23.262343 (+ 18us) client_negotiation.cc:769] Sending connection context
0925 10:54:23.262424 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 10:54:23.262575 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":366,"threads_started":1}
I20260925 10:54:23.272541 1772 consensus_queue.cc:1035] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [LEADER]: Connected to new peer: Peer: permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:23.278566 1796 mvcc.cc:205] Tried to move back new op lower bound from 7333206684759953408 to 7333206683207598080. Current Snapshot: MvccSnapshot[applied={T|T < 7333206684759953408}]
I20260925 10:54:23.288457 1696 raft_consensus.cc:1261] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Refusing update from remote peer eaefc446ad8a420cbb49ffda1e09a0be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:23.289307 1772 consensus_queue.cc:1035] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [LEADER]: Connected to new peer: Peer: permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:23.313465 1764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:23.303639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:39775 (local address 127.31.62.129:42173)
0925 10:54:23.303819 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:23.303825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:23.306746 (+ 2921us) server_negotiation.cc:407] Connection header received
0925 10:54:23.307034 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:23.307040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:23.307125 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:23.307196 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:54:23.308052 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.309258 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.311537 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.312115 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.312240 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:23.312736 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:54:23.312879 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:54:23.312956 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:23.313085 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 10:54:23.313173 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 10:54:23.313611 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:23.303187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:42173 (local address 127.31.62.131:39775)
0925 10:54:23.306630 (+ 3443us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:23.306651 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:23.306871 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:23.307388 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:23.307394 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:23.307406 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 10:54:23.307864 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:23.307877 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.309434 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.309440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:23.311369 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:23.311385 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:23.312304 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:23.312312 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:23.312485 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:23.312563 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 10:54:23.312959 (+ 396us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 10:54:23.312971 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 10:54:23.313064 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 10:54:23.313216 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2958,"thread_start_us":2822,"threads_started":1}
I20260925 10:54:23.358641 1260 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:23.359061 1260 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:23.363811 1260 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6: Sending DeleteTablet for 3 replicas of tablet 15529422f8ef4b4a94838757c49d9df0
I20260925 10:54:23.365044 1537 tablet_service.cc:1514] Processing DeleteTablet for tablet 15529422f8ef4b4a94838757c49d9df0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:23 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:48746
I20260925 10:54:23.365051 1676 tablet_service.cc:1514] Processing DeleteTablet for tablet 15529422f8ef4b4a94838757c49d9df0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:23 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:41768
I20260925 10:54:23.366277 1398 tablet_service.cc:1514] Processing DeleteTablet for tablet 15529422f8ef4b4a94838757c49d9df0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:23 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:52288
I20260925 10:54:23.369823 1259 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:54:23.370525 1259 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:54:23.377365 1817 tablet_replica.cc:330] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be: stopping tablet replica
I20260925 10:54:23.377862 1817 raft_consensus.cc:2229] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:54:23.378538 1817 raft_consensus.cc:2258] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:23.379669 1675 tablet_service.cc:1467] Processing CreateTablet for tablet 177e22b0898a48a79437d55e20546e0d (DEFAULT_TABLE table=test-table [id=14adaaa5f3c6419b93ed7e71f84b3fc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:23.380342 1675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 177e22b0898a48a79437d55e20546e0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:23.380558 1817 ts_tablet_manager.cc:1905] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:23.383996 1815 tablet_replica.cc:330] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19: stopping tablet replica
I20260925 10:54:23.384475 1815 raft_consensus.cc:2229] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:54:23.385588 1815 raft_consensus.cc:2258] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:23.385888 1398 tablet_service.cc:1467] Processing CreateTablet for tablet 177e22b0898a48a79437d55e20546e0d (DEFAULT_TABLE table=test-table [id=14adaaa5f3c6419b93ed7e71f84b3fc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:23.386304 1398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 177e22b0898a48a79437d55e20546e0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:23.387523 1815 ts_tablet_manager.cc:1905] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:23.390030 1536 tablet_service.cc:1467] Processing CreateTablet for tablet 177e22b0898a48a79437d55e20546e0d (DEFAULT_TABLE table=test-table [id=14adaaa5f3c6419b93ed7e71f84b3fc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:23.390645 1536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 177e22b0898a48a79437d55e20546e0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:23.395567 1815 ts_tablet_manager.cc:1918] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 10:54:23.395792 1815 log.cc:1196] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/15529422f8ef4b4a94838757c49d9df0
I20260925 10:54:23.396397 1815 ts_tablet_manager.cc:1939] T 15529422f8ef4b4a94838757c49d9df0 P a56110511b1f4138a7fb8255b33def19: Deleting consensus metadata
I20260925 10:54:23.396781 1765 tablet_bootstrap.cc:492] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718: Bootstrap starting.
I20260925 10:54:23.398424 1765 tablet_bootstrap.cc:654] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:23.398550 1247 catalog_manager.cc:4841] TS a56110511b1f4138a7fb8255b33def19 (127.31.62.131:33723): tablet 15529422f8ef4b4a94838757c49d9df0 (table test-table [id=5a30a8ea6e8b48588dc5444beb1c5b18]) successfully deleted
I20260925 10:54:23.400733 1765 tablet_bootstrap.cc:492] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718: No bootstrap required, opened a new log
I20260925 10:54:23.400940 1765 ts_tablet_manager.cc:1397] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 10:54:23.401687 1765 raft_consensus.cc:348] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.401957 1765 raft_consensus.cc:374] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:23.402022 1765 raft_consensus.cc:729] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73b2e052b43c450e90ad693b70710718, State: Initialized, Role: FOLLOWER
I20260925 10:54:23.402230 1767 tablet_bootstrap.cc:492] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be: Bootstrap starting.
I20260925 10:54:23.402279 1765 consensus_queue.cc:260] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.402822 1765 ts_tablet_manager.cc:1428] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260925 10:54:23.403942 1767 tablet_bootstrap.cc:654] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:23.405088 1817 ts_tablet_manager.cc:1918] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 10:54:23.405290 1817 log.cc:1196] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/15529422f8ef4b4a94838757c49d9df0
I20260925 10:54:23.405797 1817 ts_tablet_manager.cc:1939] T 15529422f8ef4b4a94838757c49d9df0 P eaefc446ad8a420cbb49ffda1e09a0be: Deleting consensus metadata
I20260925 10:54:23.409613 1246 catalog_manager.cc:4841] TS eaefc446ad8a420cbb49ffda1e09a0be (127.31.62.129:42173): tablet 15529422f8ef4b4a94838757c49d9df0 (table test-table [id=5a30a8ea6e8b48588dc5444beb1c5b18]) successfully deleted
I20260925 10:54:23.413744 1768 tablet_bootstrap.cc:492] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19: Bootstrap starting.
I20260925 10:54:23.415668 1768 tablet_bootstrap.cc:654] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:23.418990 1767 tablet_bootstrap.cc:492] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be: No bootstrap required, opened a new log
I20260925 10:54:23.419252 1767 ts_tablet_manager.cc:1397] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.006s
I20260925 10:54:23.419293 1768 tablet_bootstrap.cc:492] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19: No bootstrap required, opened a new log
I20260925 10:54:23.419450 1768 ts_tablet_manager.cc:1397] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260925 10:54:23.420125 1767 raft_consensus.cc:348] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.420210 1768 raft_consensus.cc:348] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.420461 1768 raft_consensus.cc:374] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:23.420488 1767 raft_consensus.cc:374] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:23.420547 1768 raft_consensus.cc:729] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a56110511b1f4138a7fb8255b33def19, State: Initialized, Role: FOLLOWER
I20260925 10:54:23.420663 1767 raft_consensus.cc:729] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaefc446ad8a420cbb49ffda1e09a0be, State: Initialized, Role: FOLLOWER
I20260925 10:54:23.420827 1768 consensus_queue.cc:260] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.420984 1767 consensus_queue.cc:260] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.421657 1767 ts_tablet_manager.cc:1428] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260925 10:54:23.426224 1768 ts_tablet_manager.cc:1428] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260925 10:54:23.432193 1816 tablet_replica.cc:330] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718: stopping tablet replica
I20260925 10:54:23.432708 1816 raft_consensus.cc:2229] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:54:23.433130 1816 raft_consensus.cc:2258] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:23.435092 1816 ts_tablet_manager.cc:1905] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:23.442971 1816 ts_tablet_manager.cc:1918] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 10:54:23.443186 1816 log.cc:1196] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/15529422f8ef4b4a94838757c49d9df0
I20260925 10:54:23.443833 1816 ts_tablet_manager.cc:1939] T 15529422f8ef4b4a94838757c49d9df0 P 73b2e052b43c450e90ad693b70710718: Deleting consensus metadata
I20260925 10:54:23.445958 1248 catalog_manager.cc:4841] TS 73b2e052b43c450e90ad693b70710718 (127.31.62.130:34269): tablet 15529422f8ef4b4a94838757c49d9df0 (table test-table [id=5a30a8ea6e8b48588dc5444beb1c5b18]) successfully deleted
I20260925 10:54:23.562021 1776 raft_consensus.cc:482] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:54:23.562261 1776 raft_consensus.cc:504] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.563099 1776 leader_election.cc:288] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73b2e052b43c450e90ad693b70710718 (127.31.62.130:34269), a56110511b1f4138a7fb8255b33def19 (127.31.62.131:33723)
I20260925 10:54:23.563683 1557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "177e22b0898a48a79437d55e20546e0d" candidate_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b2e052b43c450e90ad693b70710718" is_pre_election: true
I20260925 10:54:23.563951 1557 raft_consensus.cc:2454] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaefc446ad8a420cbb49ffda1e09a0be in term 0.
I20260925 10:54:23.564020 1696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "177e22b0898a48a79437d55e20546e0d" candidate_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a56110511b1f4138a7fb8255b33def19" is_pre_election: true
I20260925 10:54:23.564242 1696 raft_consensus.cc:2454] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaefc446ad8a420cbb49ffda1e09a0be in term 0.
I20260925 10:54:23.564409 1351 leader_election.cc:302] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73b2e052b43c450e90ad693b70710718, eaefc446ad8a420cbb49ffda1e09a0be; no voters:
I20260925 10:54:23.564705 1776 raft_consensus.cc:2781] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:54:23.564811 1776 raft_consensus.cc:482] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:54:23.564936 1776 raft_consensus.cc:3037] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:23.568059 1776 raft_consensus.cc:504] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.569058 1776 leader_election.cc:288] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [CANDIDATE]: Term 1 election: Requested vote from peers 73b2e052b43c450e90ad693b70710718 (127.31.62.130:34269), a56110511b1f4138a7fb8255b33def19 (127.31.62.131:33723)
I20260925 10:54:23.569278 1557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "177e22b0898a48a79437d55e20546e0d" candidate_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b2e052b43c450e90ad693b70710718"
I20260925 10:54:23.569494 1557 raft_consensus.cc:3037] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:23.572391 1557 raft_consensus.cc:2454] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaefc446ad8a420cbb49ffda1e09a0be in term 1.
I20260925 10:54:23.572994 1351 leader_election.cc:302] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73b2e052b43c450e90ad693b70710718, eaefc446ad8a420cbb49ffda1e09a0be; no voters:
I20260925 10:54:23.573208 1696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "177e22b0898a48a79437d55e20546e0d" candidate_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a56110511b1f4138a7fb8255b33def19"
I20260925 10:54:23.573328 1776 raft_consensus.cc:2781] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:23.573422 1696 raft_consensus.cc:3037] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:23.573777 1776 raft_consensus.cc:686] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [term 1 LEADER]: Becoming Leader. State: Replica: eaefc446ad8a420cbb49ffda1e09a0be, State: Running, Role: LEADER
I20260925 10:54:23.574070 1776 consensus_queue.cc:237] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:23.576578 1696 raft_consensus.cc:2454] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaefc446ad8a420cbb49ffda1e09a0be in term 1.
I20260925 10:54:23.579668 1259 catalog_manager.cc:5495] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be reported cstate change: term changed from 0 to 1, leader changed from <none> to eaefc446ad8a420cbb49ffda1e09a0be (127.31.62.129). New cstate: current_term: 1 leader_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } health_report { overall_health: UNKNOWN } } }
I20260925 10:54:23.616025 1696 raft_consensus.cc:1261] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Refusing update from remote peer eaefc446ad8a420cbb49ffda1e09a0be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:23.616276 1557 raft_consensus.cc:1261] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Refusing update from remote peer eaefc446ad8a420cbb49ffda1e09a0be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:23.616825 1772 consensus_queue.cc:1035] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [LEADER]: Connected to new peer: Peer: permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:23.618696 1776 consensus_queue.cc:1035] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [LEADER]: Connected to new peer: Peer: permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 10:54:23.669054 1796 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 10:54:25.261015 1259 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:25.261364 1259 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:25.265631 1259 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6: Sending DeleteTablet for 3 replicas of tablet 177e22b0898a48a79437d55e20546e0d
I20260925 10:54:25.266525 1537 tablet_service.cc:1514] Processing DeleteTablet for tablet 177e22b0898a48a79437d55e20546e0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:25 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:48746
I20260925 10:54:25.266525 1398 tablet_service.cc:1514] Processing DeleteTablet for tablet 177e22b0898a48a79437d55e20546e0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:25 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:52288
I20260925 10:54:25.266587 1675 tablet_service.cc:1514] Processing DeleteTablet for tablet 177e22b0898a48a79437d55e20546e0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:25 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:41768
I20260925 10:54:25.286850 1847 tablet_replica.cc:330] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718: stopping tablet replica
I20260925 10:54:25.287276 1847 raft_consensus.cc:2229] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:54:25.287613 1847 raft_consensus.cc:2258] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:25.288589 1848 tablet_replica.cc:330] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be: stopping tablet replica
I20260925 10:54:25.288779 1848 raft_consensus.cc:2229] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:54:25.289409 1849 tablet_replica.cc:330] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19: stopping tablet replica
I20260925 10:54:25.289453 1847 ts_tablet_manager.cc:1905] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:25.289577 1849 raft_consensus.cc:2229] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:54:25.290253 1849 raft_consensus.cc:2258] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:25.291158 1849 ts_tablet_manager.cc:1905] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:25.291131 1848 raft_consensus.cc:2258] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:25.292084 1848 ts_tablet_manager.cc:1905] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 25 10:54:25 dist-test-slave-nx7s krb5kdc[1216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333665, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260925 10:54:25.386209 31994 mini_kdc.cc:317] Time spent kinit for test-user: real 0.119s user 0.000s sys 0.012s
I20260925 10:54:25.390319 1847 ts_tablet_manager.cc:1918] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 10:54:25.390473 1847 log.cc:1196] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/177e22b0898a48a79437d55e20546e0d
I20260925 10:54:25.390919 1847 ts_tablet_manager.cc:1939] T 177e22b0898a48a79437d55e20546e0d P 73b2e052b43c450e90ad693b70710718: Deleting consensus metadata
I20260925 10:54:25.391067 1849 ts_tablet_manager.cc:1918] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 10:54:25.391202 1849 log.cc:1196] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/177e22b0898a48a79437d55e20546e0d
I20260925 10:54:25.391674 1849 ts_tablet_manager.cc:1939] T 177e22b0898a48a79437d55e20546e0d P a56110511b1f4138a7fb8255b33def19: Deleting consensus metadata
Sep 25 10:54:25 dist-test-slave-nx7s krb5kdc[1216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333665, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:25.407112 1857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:25.392442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55276 (local address 127.31.62.190:44035)
0925 10:54:25.393084 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:25.393089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:25.393120 (+ 31us) server_negotiation.cc:407] Connection header received
0925 10:54:25.393309 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:25.393314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:25.393394 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:25.393496 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:25.394880 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.395658 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.397114 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.397639 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.397755 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:25.401794 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:25.401813 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:25.401816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:25.401854 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:25.404456 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:25.405202 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:25.405206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:25.405208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:25.405333 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:25.405680 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:25.405683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:25.405685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:25.406051 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:25.406256 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:25.406560 (+ 304us) server_negotiation.cc:299] Negotiation successful
0925 10:54:25.406751 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":275,"threads_started":1}
I20260925 10:54:25.410621 1259 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:54:25.411167 1259 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:54:25.418005 1398 tablet_service.cc:1467] Processing CreateTablet for tablet f609716c8ab04d12b8d895f22eabacf1 (DEFAULT_TABLE table=test-table [id=b4d915d5003e4885a76166a047d54dea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:25.418114 1537 tablet_service.cc:1467] Processing CreateTablet for tablet f609716c8ab04d12b8d895f22eabacf1 (DEFAULT_TABLE table=test-table [id=b4d915d5003e4885a76166a047d54dea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:25.418387 1675 tablet_service.cc:1467] Processing CreateTablet for tablet f609716c8ab04d12b8d895f22eabacf1 (DEFAULT_TABLE table=test-table [id=b4d915d5003e4885a76166a047d54dea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:25.418524 1537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f609716c8ab04d12b8d895f22eabacf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:25.418769 1675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f609716c8ab04d12b8d895f22eabacf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:25.425518 1858 tablet_bootstrap.cc:492] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19: Bootstrap starting.
I20260925 10:54:25.426152 1859 tablet_bootstrap.cc:492] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718: Bootstrap starting.
I20260925 10:54:25.427233 1858 tablet_bootstrap.cc:654] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:25.427772 1859 tablet_bootstrap.cc:654] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:25.430020 1858 tablet_bootstrap.cc:492] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19: No bootstrap required, opened a new log
I20260925 10:54:25.430020 1859 tablet_bootstrap.cc:492] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718: No bootstrap required, opened a new log
I20260925 10:54:25.430172 1859 ts_tablet_manager.cc:1397] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 10:54:25.430172 1858 ts_tablet_manager.cc:1397] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 10:54:25.430932 1858 raft_consensus.cc:348] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:25.430933 1859 raft_consensus.cc:348] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:25.431186 1859 raft_consensus.cc:374] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:25.431180 1858 raft_consensus.cc:374] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:25.431278 1859 raft_consensus.cc:729] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73b2e052b43c450e90ad693b70710718, State: Initialized, Role: FOLLOWER
I20260925 10:54:25.431278 1858 raft_consensus.cc:729] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a56110511b1f4138a7fb8255b33def19, State: Initialized, Role: FOLLOWER
I20260925 10:54:25.431533 1859 consensus_queue.cc:260] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:25.431532 1858 consensus_queue.cc:260] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:25.432217 1858 ts_tablet_manager.cc:1428] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 10:54:25.432819 1859 ts_tablet_manager.cc:1428] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 10:54:25.480237 1247 catalog_manager.cc:4841] TS a56110511b1f4138a7fb8255b33def19 (127.31.62.131:33723): tablet 177e22b0898a48a79437d55e20546e0d (table test-table [id=14adaaa5f3c6419b93ed7e71f84b3fc0]) successfully deleted
I20260925 10:54:25.480259 1248 catalog_manager.cc:4841] TS 73b2e052b43c450e90ad693b70710718 (127.31.62.130:34269): tablet 177e22b0898a48a79437d55e20546e0d (table test-table [id=14adaaa5f3c6419b93ed7e71f84b3fc0]) successfully deleted
I20260925 10:54:25.487282 1398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f609716c8ab04d12b8d895f22eabacf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:25.487293 1848 ts_tablet_manager.cc:1918] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 10:54:25.487495 1848 log.cc:1196] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/177e22b0898a48a79437d55e20546e0d
I20260925 10:54:25.487962 1848 ts_tablet_manager.cc:1939] T 177e22b0898a48a79437d55e20546e0d P eaefc446ad8a420cbb49ffda1e09a0be: Deleting consensus metadata
I20260925 10:54:25.489681 1246 catalog_manager.cc:4841] TS eaefc446ad8a420cbb49ffda1e09a0be (127.31.62.129:42173): tablet 177e22b0898a48a79437d55e20546e0d (table test-table [id=14adaaa5f3c6419b93ed7e71f84b3fc0]) successfully deleted
I20260925 10:54:25.492815 1863 tablet_bootstrap.cc:492] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be: Bootstrap starting.
I20260925 10:54:25.494582 1863 tablet_bootstrap.cc:654] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:25.496347 1863 tablet_bootstrap.cc:492] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be: No bootstrap required, opened a new log
I20260925 10:54:25.496513 1863 ts_tablet_manager.cc:1397] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 10:54:25.497211 1863 raft_consensus.cc:348] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:25.497400 1863 raft_consensus.cc:374] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:25.497488 1863 raft_consensus.cc:729] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaefc446ad8a420cbb49ffda1e09a0be, State: Initialized, Role: FOLLOWER
I20260925 10:54:25.497726 1863 consensus_queue.cc:260] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:25.498277 1863 ts_tablet_manager.cc:1428] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260925 10:54:25.637044 1862 raft_consensus.cc:482] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:54:25.637324 1862 raft_consensus.cc:504] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:25.640579 1862 leader_election.cc:288] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eaefc446ad8a420cbb49ffda1e09a0be (127.31.62.129:42173), a56110511b1f4138a7fb8255b33def19 (127.31.62.131:33723)
I20260925 10:54:25.657641 1833 raft_consensus.cc:482] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:54:25.657821 1833 raft_consensus.cc:504] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:25.658471 1833 leader_election.cc:288] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eaefc446ad8a420cbb49ffda1e09a0be (127.31.62.129:42173), 73b2e052b43c450e90ad693b70710718 (127.31.62.130:34269)
I20260925 10:54:25.658962 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:25.640302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:33723 (local address 127.31.62.130:57379)
0925 10:54:25.650689 (+ 10387us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:25.650714 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:25.650902 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:25.651703 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:25.651707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:25.651744 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:25.652148 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:25.652156 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.653460 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.653466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:25.656230 (+ 2764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:25.656248 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.658082 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.658089 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:25.658269 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:25.658302 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 10:54:25.658408 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 10:54:25.658552 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10221,"mutex_wait_us":24,"thread_start_us":10058,"threads_started":1}
I20260925 10:54:25.658916 1418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f609716c8ab04d12b8d895f22eabacf1" candidate_uuid: "a56110511b1f4138a7fb8255b33def19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" is_pre_election: true
I20260925 10:54:25.659160 1418 raft_consensus.cc:2454] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a56110511b1f4138a7fb8255b33def19 in term 0.
I20260925 10:54:25.659235 1866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:25.640342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:40205 (local address 127.31.62.129:42173)
0925 10:54:25.650420 (+ 10078us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:25.650426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:25.651257 (+ 831us) server_negotiation.cc:407] Connection header received
0925 10:54:25.651344 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:25.651350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:25.651421 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:25.651566 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:25.652271 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.653455 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.656775 (+ 3320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.658001 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.658224 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:25.658293 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:25.658763 (+ 470us) server_negotiation.cc:299] Negotiation successful
0925 10:54:25.658906 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9911,"thread_start_us":9723,"threads_started":1}
I20260925 10:54:25.659412 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:25.640375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:42173 (local address 127.31.62.130:40205)
0925 10:54:25.650969 (+ 10594us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:25.650988 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:25.651149 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:25.651603 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:25.651608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:25.651631 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:25.652062 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:25.652078 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.653595 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.653599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:25.656264 (+ 2665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:25.656274 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.658438 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.658444 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:25.658596 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:25.658623 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 10:54:25.658711 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 10:54:25.658855 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10471,"mutex_wait_us":40,"thread_start_us":10317,"threads_started":1}
I20260925 10:54:25.659627 1628 leader_election.cc:302] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a56110511b1f4138a7fb8255b33def19, eaefc446ad8a420cbb49ffda1e09a0be; no voters:
I20260925 10:54:25.659914 1833 raft_consensus.cc:2781] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:54:25.660032 1833 raft_consensus.cc:482] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:54:25.660043 1557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f609716c8ab04d12b8d895f22eabacf1" candidate_uuid: "a56110511b1f4138a7fb8255b33def19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b2e052b43c450e90ad693b70710718" is_pre_election: true
I20260925 10:54:25.660138 1833 raft_consensus.cc:3037] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:25.660260 1557 raft_consensus.cc:2454] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a56110511b1f4138a7fb8255b33def19 in term 0.
I20260925 10:54:25.660218 1418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f609716c8ab04d12b8d895f22eabacf1" candidate_uuid: "73b2e052b43c450e90ad693b70710718" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" is_pre_election: true
I20260925 10:54:25.660420 1418 raft_consensus.cc:2454] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73b2e052b43c450e90ad693b70710718 in term 0.
I20260925 10:54:25.660620 1868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:25.640881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:57379 (local address 127.31.62.131:33723)
0925 10:54:25.651427 (+ 10546us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:25.651431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:25.651450 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:54:25.651504 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:25.651507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:25.651566 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:25.652283 (+ 717us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:25.652457 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.653334 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.656414 (+ 3080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.657683 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.657862 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:25.657940 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:25.659044 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0925 10:54:25.659747 (+ 703us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10443,"thread_start_us":10298,"threads_started":1}
I20260925 10:54:25.660943 1489 leader_election.cc:302] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73b2e052b43c450e90ad693b70710718, eaefc446ad8a420cbb49ffda1e09a0be; no voters:
I20260925 10:54:25.661245 1696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f609716c8ab04d12b8d895f22eabacf1" candidate_uuid: "73b2e052b43c450e90ad693b70710718" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a56110511b1f4138a7fb8255b33def19" is_pre_election: true
I20260925 10:54:25.661394 1862 raft_consensus.cc:2781] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:54:25.661516 1862 raft_consensus.cc:482] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:54:25.661620 1862 raft_consensus.cc:3037] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:25.691314 1862 raft_consensus.cc:504] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:25.691314 1833 raft_consensus.cc:504] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:25.691895 1696 raft_consensus.cc:2379] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 73b2e052b43c450e90ad693b70710718 in current term 1: Already voted for candidate a56110511b1f4138a7fb8255b33def19 in this term.
I20260925 10:54:25.692039 1833 leader_election.cc:288] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [CANDIDATE]: Term 1 election: Requested vote from peers eaefc446ad8a420cbb49ffda1e09a0be (127.31.62.129:42173), 73b2e052b43c450e90ad693b70710718 (127.31.62.130:34269)
I20260925 10:54:25.692735 1418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f609716c8ab04d12b8d895f22eabacf1" candidate_uuid: "73b2e052b43c450e90ad693b70710718" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefc446ad8a420cbb49ffda1e09a0be"
I20260925 10:54:25.693027 1418 raft_consensus.cc:3037] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:25.693095 1417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f609716c8ab04d12b8d895f22eabacf1" candidate_uuid: "a56110511b1f4138a7fb8255b33def19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefc446ad8a420cbb49ffda1e09a0be"
I20260925 10:54:25.694087 1557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f609716c8ab04d12b8d895f22eabacf1" candidate_uuid: "a56110511b1f4138a7fb8255b33def19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b2e052b43c450e90ad693b70710718"
I20260925 10:54:25.694301 1696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f609716c8ab04d12b8d895f22eabacf1" candidate_uuid: "73b2e052b43c450e90ad693b70710718" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a56110511b1f4138a7fb8255b33def19"
I20260925 10:54:25.694396 1557 raft_consensus.cc:2379] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a56110511b1f4138a7fb8255b33def19 in current term 1: Already voted for candidate 73b2e052b43c450e90ad693b70710718 in this term.
I20260925 10:54:25.694527 1696 raft_consensus.cc:2379] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 73b2e052b43c450e90ad693b70710718 in current term 1: Already voted for candidate a56110511b1f4138a7fb8255b33def19 in this term.
I20260925 10:54:25.694097 1862 leader_election.cc:288] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [CANDIDATE]: Term 1 election: Requested vote from peers eaefc446ad8a420cbb49ffda1e09a0be (127.31.62.129:42173), a56110511b1f4138a7fb8255b33def19 (127.31.62.131:33723)
I20260925 10:54:25.695035 1630 leader_election.cc:302] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a56110511b1f4138a7fb8255b33def19; no voters: 73b2e052b43c450e90ad693b70710718, eaefc446ad8a420cbb49ffda1e09a0be
I20260925 10:54:25.695355 1833 raft_consensus.cc:2726] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 10:54:25.795632 1418 raft_consensus.cc:2454] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73b2e052b43c450e90ad693b70710718 in term 1.
I20260925 10:54:25.796267 1489 leader_election.cc:302] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73b2e052b43c450e90ad693b70710718, eaefc446ad8a420cbb49ffda1e09a0be; no voters: a56110511b1f4138a7fb8255b33def19
I20260925 10:54:25.796609 1862 raft_consensus.cc:2781] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:25.797029 1862 raft_consensus.cc:686] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [term 1 LEADER]: Becoming Leader. State: Replica: 73b2e052b43c450e90ad693b70710718, State: Running, Role: LEADER
I20260925 10:54:25.797477 1862 consensus_queue.cc:237] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:25.801512 1259 catalog_manager.cc:5495] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 reported cstate change: term changed from 0 to 1, leader changed from <none> to 73b2e052b43c450e90ad693b70710718 (127.31.62.130). New cstate: current_term: 1 leader_uuid: "73b2e052b43c450e90ad693b70710718" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } health_report { overall_health: UNKNOWN } } }
I20260925 10:54:25.895936 1879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:25.890236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45708 (local address 127.31.62.130:34269)
0925 10:54:25.890773 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:25.890778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:25.890795 (+ 17us) server_negotiation.cc:407] Connection header received
0925 10:54:25.890879 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:25.890882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:25.890958 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:25.891112 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:54:25.891564 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.892349 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.893708 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.894140 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.894251 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:25.894710 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:54:25.895286 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:54:25.895413 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:25.895514 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 10:54:25.895592 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":261,"threads_started":1}
I20260925 10:54:25.912619 1868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:25.907132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:40085 (local address 127.31.62.131:33723)
0925 10:54:25.907349 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:25.907355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:25.907894 (+ 539us) server_negotiation.cc:407] Connection header received
0925 10:54:25.908060 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:25.908064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:25.908133 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:25.908220 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:54:25.908759 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.909558 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.911001 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.911418 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.911517 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:25.911976 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:54:25.912088 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:54:25.912151 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:25.912359 (+ 208us) server_negotiation.cc:299] Negotiation successful
0925 10:54:25.912423 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 10:54:25.912791 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:25.907115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:33723 (local address 127.31.62.130:40085)
0925 10:54:25.907791 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:25.907808 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:25.907939 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:25.908288 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:25.908291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:25.908303 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 10:54:25.908592 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:25.908604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.909694 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.909699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:25.910857 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:25.910871 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.911610 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.911616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:25.911754 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:25.911812 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 10:54:25.912197 (+ 385us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 10:54:25.912210 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 10:54:25.912314 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 10:54:25.912429 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":336,"threads_started":1}
I20260925 10:54:25.921003 1418 raft_consensus.cc:1261] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Refusing update from remote peer 73b2e052b43c450e90ad693b70710718: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:25.921015 1696 raft_consensus.cc:1261] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Refusing update from remote peer 73b2e052b43c450e90ad693b70710718: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:25.921681 1862 consensus_queue.cc:1035] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:25.921973 1869 consensus_queue.cc:1035] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:25.949892 1879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:25.944918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:48779 (local address 127.31.62.130:34269)
0925 10:54:25.945073 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:25.945079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:25.945294 (+ 215us) server_negotiation.cc:407] Connection header received
0925 10:54:25.945401 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:25.945404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:25.945469 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:25.945547 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:54:25.945815 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.946671 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.948324 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.948793 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.948943 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:25.949338 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:54:25.949436 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:54:25.949497 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:25.949609 (+ 112us) server_negotiation.cc:299] Negotiation successful
0925 10:54:25.949680 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 10:54:25.949925 1887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:25.944636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:34269 (local address 127.31.62.131:48779)
0925 10:54:25.945161 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:25.945177 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:25.945313 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:25.945555 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:25.945558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:25.945566 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 10:54:25.945726 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:25.945732 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.946830 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.946836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:25.947993 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:25.948009 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:25.948974 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:25.948979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:25.949111 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:25.949149 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 10:54:25.949516 (+ 367us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 10:54:25.949525 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 10:54:25.949586 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 10:54:25.949696 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":341,"threads_started":1}
I20260925 10:54:25.984393 1260 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:25.984629 1260 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:25.988029 1260 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6: Sending DeleteTablet for 3 replicas of tablet f609716c8ab04d12b8d895f22eabacf1
I20260925 10:54:25.988957 1398 tablet_service.cc:1514] Processing DeleteTablet for tablet f609716c8ab04d12b8d895f22eabacf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:25 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:52288
I20260925 10:54:25.989060 1675 tablet_service.cc:1514] Processing DeleteTablet for tablet f609716c8ab04d12b8d895f22eabacf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:25 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:41768
I20260925 10:54:25.989037 1537 tablet_service.cc:1514] Processing DeleteTablet for tablet f609716c8ab04d12b8d895f22eabacf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:25 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:48746
I20260925 10:54:25.989277 1848 tablet_replica.cc:330] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be: stopping tablet replica
I20260925 10:54:25.989468 1848 raft_consensus.cc:2229] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:54:25.989658 1848 raft_consensus.cc:2258] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:25.991060 1848 ts_tablet_manager.cc:1905] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:25.995620 1892 tablet_replica.cc:330] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718: stopping tablet replica
I20260925 10:54:25.996120 1891 tablet_replica.cc:330] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19: stopping tablet replica
I20260925 10:54:25.996539 1260 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:54:25.997318 1260 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:54:26.000003 1848 ts_tablet_manager.cc:1918] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 10:54:26.000172 1848 log.cc:1196] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/f609716c8ab04d12b8d895f22eabacf1
I20260925 10:54:26.000563 1848 ts_tablet_manager.cc:1939] T f609716c8ab04d12b8d895f22eabacf1 P eaefc446ad8a420cbb49ffda1e09a0be: Deleting consensus metadata
I20260925 10:54:26.000926 1891 raft_consensus.cc:2229] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:54:26.001214 1891 raft_consensus.cc:2258] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:26.001582 1892 raft_consensus.cc:2229] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:54:26.002192 1892 raft_consensus.cc:2258] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:26.003010 1891 ts_tablet_manager.cc:1905] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:26.004778 1398 tablet_service.cc:1467] Processing CreateTablet for tablet 4bdaf9ee8914453f826c1aeb8abef8e1 (DEFAULT_TABLE table=test-table [id=31601b94cfff453c88d85832f628550e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:26.005323 1398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bdaf9ee8914453f826c1aeb8abef8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:26.005596 1675 tablet_service.cc:1467] Processing CreateTablet for tablet 4bdaf9ee8914453f826c1aeb8abef8e1 (DEFAULT_TABLE table=test-table [id=31601b94cfff453c88d85832f628550e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:26.006181 1675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bdaf9ee8914453f826c1aeb8abef8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:26.008061 1892 ts_tablet_manager.cc:1905] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:26.011220 1537 tablet_service.cc:1467] Processing CreateTablet for tablet 4bdaf9ee8914453f826c1aeb8abef8e1 (DEFAULT_TABLE table=test-table [id=31601b94cfff453c88d85832f628550e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:26.013530 1893 tablet_bootstrap.cc:492] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be: Bootstrap starting.
I20260925 10:54:26.015362 1893 tablet_bootstrap.cc:654] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:26.022132 1893 tablet_bootstrap.cc:492] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be: No bootstrap required, opened a new log
I20260925 10:54:26.022330 1893 ts_tablet_manager.cc:1397] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260925 10:54:26.023150 1893 raft_consensus.cc:348] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:26.023424 1893 raft_consensus.cc:374] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:26.023510 1893 raft_consensus.cc:729] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaefc446ad8a420cbb49ffda1e09a0be, State: Initialized, Role: FOLLOWER
I20260925 10:54:26.023741 1893 consensus_queue.cc:260] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:26.024369 1893 ts_tablet_manager.cc:1428] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260925 10:54:26.071583 1246 catalog_manager.cc:4841] TS eaefc446ad8a420cbb49ffda1e09a0be (127.31.62.129:42173): tablet f609716c8ab04d12b8d895f22eabacf1 (table test-table [id=b4d915d5003e4885a76166a047d54dea]) successfully deleted
I20260925 10:54:26.098275 1537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bdaf9ee8914453f826c1aeb8abef8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:26.106730 1895 tablet_bootstrap.cc:492] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19: Bootstrap starting.
I20260925 10:54:26.109126 1895 tablet_bootstrap.cc:654] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:26.111585 1895 tablet_bootstrap.cc:492] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19: No bootstrap required, opened a new log
I20260925 10:54:26.111750 1895 ts_tablet_manager.cc:1397] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260925 10:54:26.112550 1895 raft_consensus.cc:348] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:26.112802 1895 raft_consensus.cc:374] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:26.112948 1895 raft_consensus.cc:729] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a56110511b1f4138a7fb8255b33def19, State: Initialized, Role: FOLLOWER
I20260925 10:54:26.113205 1895 consensus_queue.cc:260] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:26.113797 1895 ts_tablet_manager.cc:1428] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 10:54:26.200183 1892 ts_tablet_manager.cc:1918] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 10:54:26.200183 1891 ts_tablet_manager.cc:1918] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 10:54:26.200380 1892 log.cc:1196] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/f609716c8ab04d12b8d895f22eabacf1
I20260925 10:54:26.200526 1891 log.cc:1196] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/f609716c8ab04d12b8d895f22eabacf1
I20260925 10:54:26.200956 1892 ts_tablet_manager.cc:1939] T f609716c8ab04d12b8d895f22eabacf1 P 73b2e052b43c450e90ad693b70710718: Deleting consensus metadata
I20260925 10:54:26.201102 1891 ts_tablet_manager.cc:1939] T f609716c8ab04d12b8d895f22eabacf1 P a56110511b1f4138a7fb8255b33def19: Deleting consensus metadata
I20260925 10:54:26.202172 1248 catalog_manager.cc:4841] TS 73b2e052b43c450e90ad693b70710718 (127.31.62.130:34269): tablet f609716c8ab04d12b8d895f22eabacf1 (table test-table [id=b4d915d5003e4885a76166a047d54dea]) successfully deleted
I20260925 10:54:26.202701 1247 catalog_manager.cc:4841] TS a56110511b1f4138a7fb8255b33def19 (127.31.62.131:33723): tablet f609716c8ab04d12b8d895f22eabacf1 (table test-table [id=b4d915d5003e4885a76166a047d54dea]) successfully deleted
I20260925 10:54:26.206516 1897 tablet_bootstrap.cc:492] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718: Bootstrap starting.
I20260925 10:54:26.208827 1897 tablet_bootstrap.cc:654] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:26.211128 1897 tablet_bootstrap.cc:492] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718: No bootstrap required, opened a new log
I20260925 10:54:26.211293 1897 ts_tablet_manager.cc:1397] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260925 10:54:26.211969 1897 raft_consensus.cc:348] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:26.212177 1897 raft_consensus.cc:374] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:26.212282 1897 raft_consensus.cc:729] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73b2e052b43c450e90ad693b70710718, State: Initialized, Role: FOLLOWER
I20260925 10:54:26.212527 1897 consensus_queue.cc:260] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:26.213060 1897 ts_tablet_manager.cc:1428] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 10:54:26.352147 1889 raft_consensus.cc:482] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:54:26.352398 1889 raft_consensus.cc:504] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:26.353389 1889 leader_election.cc:288] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eaefc446ad8a420cbb49ffda1e09a0be (127.31.62.129:42173), 73b2e052b43c450e90ad693b70710718 (127.31.62.130:34269)
I20260925 10:54:26.354266 1418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bdaf9ee8914453f826c1aeb8abef8e1" candidate_uuid: "a56110511b1f4138a7fb8255b33def19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" is_pre_election: true
I20260925 10:54:26.354270 1557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bdaf9ee8914453f826c1aeb8abef8e1" candidate_uuid: "a56110511b1f4138a7fb8255b33def19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b2e052b43c450e90ad693b70710718" is_pre_election: true
I20260925 10:54:26.354560 1418 raft_consensus.cc:2454] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a56110511b1f4138a7fb8255b33def19 in term 0.
I20260925 10:54:26.354592 1557 raft_consensus.cc:2454] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a56110511b1f4138a7fb8255b33def19 in term 0.
I20260925 10:54:26.355094 1628 leader_election.cc:302] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a56110511b1f4138a7fb8255b33def19, eaefc446ad8a420cbb49ffda1e09a0be; no voters:
I20260925 10:54:26.355406 1889 raft_consensus.cc:2781] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:54:26.355535 1889 raft_consensus.cc:482] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:54:26.355645 1889 raft_consensus.cc:3037] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:26.402693 1889 raft_consensus.cc:504] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:26.403537 1889 leader_election.cc:288] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [CANDIDATE]: Term 1 election: Requested vote from peers eaefc446ad8a420cbb49ffda1e09a0be (127.31.62.129:42173), 73b2e052b43c450e90ad693b70710718 (127.31.62.130:34269)
I20260925 10:54:26.404194 1418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bdaf9ee8914453f826c1aeb8abef8e1" candidate_uuid: "a56110511b1f4138a7fb8255b33def19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefc446ad8a420cbb49ffda1e09a0be"
I20260925 10:54:26.404429 1418 raft_consensus.cc:3037] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:26.404631 1557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bdaf9ee8914453f826c1aeb8abef8e1" candidate_uuid: "a56110511b1f4138a7fb8255b33def19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b2e052b43c450e90ad693b70710718"
I20260925 10:54:26.404963 1557 raft_consensus.cc:3037] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:26.507169 1557 raft_consensus.cc:2454] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a56110511b1f4138a7fb8255b33def19 in term 1.
I20260925 10:54:26.507181 1418 raft_consensus.cc:2454] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a56110511b1f4138a7fb8255b33def19 in term 1.
I20260925 10:54:26.507822 1628 leader_election.cc:302] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a56110511b1f4138a7fb8255b33def19, eaefc446ad8a420cbb49ffda1e09a0be; no voters:
I20260925 10:54:26.508167 1889 raft_consensus.cc:2781] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:26.508605 1889 raft_consensus.cc:686] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [term 1 LEADER]: Becoming Leader. State: Replica: a56110511b1f4138a7fb8255b33def19, State: Running, Role: LEADER
I20260925 10:54:26.508890 1889 consensus_queue.cc:237] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } }
I20260925 10:54:26.511670 1260 catalog_manager.cc:5495] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 reported cstate change: term changed from 0 to 1, leader changed from <none> to a56110511b1f4138a7fb8255b33def19 (127.31.62.131). New cstate: current_term: 1 leader_uuid: "a56110511b1f4138a7fb8255b33def19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a56110511b1f4138a7fb8255b33def19" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 33723 } health_report { overall_health: HEALTHY } } }
I20260925 10:54:26.600755 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:26.595356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48396 (local address 127.31.62.131:33723)
0925 10:54:26.595916 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:26.595921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:26.596049 (+ 128us) server_negotiation.cc:407] Connection header received
0925 10:54:26.596233 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:26.596236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:26.596314 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:26.596391 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:54:26.596885 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:26.597641 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:26.599100 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:26.599548 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:26.599643 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:26.600066 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:54:26.600167 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:54:26.600231 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:26.600352 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 10:54:26.600436 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":255,"threads_started":1}
I20260925 10:54:26.616393 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:26.609662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:55667 (local address 127.31.62.131:33723)
0925 10:54:26.609831 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:26.609837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:26.610306 (+ 469us) server_negotiation.cc:407] Connection header received
0925 10:54:26.610513 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:26.610518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:26.610580 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:26.610656 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:54:26.611282 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:26.612319 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:26.614736 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:26.615155 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:26.615315 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:26.615753 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:54:26.615870 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:54:26.615938 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:26.616070 (+ 132us) server_negotiation.cc:299] Negotiation successful
0925 10:54:26.616164 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":66}
I20260925 10:54:26.616683 1916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:26.609423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:33723 (local address 127.31.62.131:55667)
0925 10:54:26.610213 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:26.610230 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:26.610399 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:26.610826 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:26.610830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:26.610840 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 10:54:26.611132 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:26.611146 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:26.612473 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:26.612479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:26.614559 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:26.614577 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:26.615340 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:26.615346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:26.615530 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:26.615578 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 10:54:26.615963 (+ 385us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 10:54:26.615974 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 10:54:26.616055 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 10:54:26.616269 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":51,"thread_start_us":529,"threads_started":1}
I20260925 10:54:26.625082 1557 raft_consensus.cc:1261] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Refusing update from remote peer a56110511b1f4138a7fb8255b33def19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:26.625195 1418 raft_consensus.cc:1261] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Refusing update from remote peer a56110511b1f4138a7fb8255b33def19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:26.625651 1908 consensus_queue.cc:1035] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73b2e052b43c450e90ad693b70710718" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:26.626053 1908 consensus_queue.cc:1035] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaefc446ad8a420cbb49ffda1e09a0be" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:26.638319 1905 mvcc.cc:205] Tried to move back new op lower bound from 7333206698492071936 to 7333206698022645760. Current Snapshot: MvccSnapshot[applied={T|T < 7333206698492071936}]
W20260925 10:54:26.654219 1904 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 10:54:28.258948 1260 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:28.259254 1260 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:28.264199 1260 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c74bb658ce914e448f49031e2e079fa6: Sending DeleteTablet for 3 replicas of tablet 4bdaf9ee8914453f826c1aeb8abef8e1
I20260925 10:54:28.265069 1537 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bdaf9ee8914453f826c1aeb8abef8e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:28 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:48746
I20260925 10:54:28.265126 1398 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bdaf9ee8914453f826c1aeb8abef8e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:28 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:52288
I20260925 10:54:28.265630 1676 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bdaf9ee8914453f826c1aeb8abef8e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:28 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:41768
I20260925 10:54:28.266119 1942 tablet_replica.cc:330] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718: stopping tablet replica
I20260925 10:54:28.266316 1942 raft_consensus.cc:2229] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:54:28.266587 1942 raft_consensus.cc:2258] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:28.267122 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 1332
I20260925 10:54:28.267607 1942 ts_tablet_manager.cc:1905] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:28.273264 1943 tablet_replica.cc:330] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19: stopping tablet replica
I20260925 10:54:28.273520 1943 raft_consensus.cc:2229] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:54:28.274170 1943 raft_consensus.cc:2258] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:28.274230 1941 tablet_replica.cc:330] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be: stopping tablet replica
I20260925 10:54:28.274435 1941 raft_consensus.cc:2229] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:54:28.274654 1941 raft_consensus.cc:2258] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:28.275279 1943 ts_tablet_manager.cc:1905] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:28.275496 1941 ts_tablet_manager.cc:1905] T 4bdaf9ee8914453f826c1aeb8abef8e1 P eaefc446ad8a420cbb49ffda1e09a0be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:28.286602 1942 ts_tablet_manager.cc:1918] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 10:54:28.286887 1942 log.cc:1196] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/4bdaf9ee8914453f826c1aeb8abef8e1
I20260925 10:54:28.286882 1943 ts_tablet_manager.cc:1918] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 10:54:28.287015 1943 log.cc:1196] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/4bdaf9ee8914453f826c1aeb8abef8e1
I20260925 10:54:28.287488 1943 ts_tablet_manager.cc:1939] T 4bdaf9ee8914453f826c1aeb8abef8e1 P a56110511b1f4138a7fb8255b33def19: Deleting consensus metadata
I20260925 10:54:28.287602 1942 ts_tablet_manager.cc:1939] T 4bdaf9ee8914453f826c1aeb8abef8e1 P 73b2e052b43c450e90ad693b70710718: Deleting consensus metadata
Sep 25 10:54:28 dist-test-slave-nx7s krb5kdc[1216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333665, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:54:28.295146 1945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:28.268573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60410 (local address 127.31.62.129:42173)
0925 10:54:28.274038 (+ 5465us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:28.274043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:28.274068 (+ 25us) server_negotiation.cc:407] Connection header received
0925 10:54:28.274131 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:28.274134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:28.274218 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:28.274331 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:28.275904 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:28.276670 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:28.278000 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:28.278489 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:28.278597 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:28.288979 (+ 10382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:28.289001 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:28.289024 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:28.289075 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:28.291962 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:28.292803 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:28.292812 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:28.292815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:28.293041 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:28.293537 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:28.293542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:28.293544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:28.294015 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:28.294256 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:28.294644 (+ 388us) server_negotiation.cc:299] Negotiation successful
0925 10:54:28.294870 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5285,"thread_start_us":5112,"threads_started":1}
W20260925 10:54:28.295837 1462 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60410
W20260925 10:54:28.296572 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 1332: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:54:28.296759 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 1332
I20260925 10:54:28.357559 1248 catalog_manager.cc:4841] TS 73b2e052b43c450e90ad693b70710718 (127.31.62.130:34269): tablet 4bdaf9ee8914453f826c1aeb8abef8e1 (table test-table [id=31601b94cfff453c88d85832f628550e]) successfully deleted
I20260925 10:54:28.370609 1247 catalog_manager.cc:4841] TS a56110511b1f4138a7fb8255b33def19 (127.31.62.131:33723): tablet 4bdaf9ee8914453f826c1aeb8abef8e1 (table test-table [id=31601b94cfff453c88d85832f628550e]) successfully deleted
W20260925 10:54:28.398514 1246 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.62.129:42173 (error 108)
I20260925 10:54:28.399360 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 1472
I20260925 10:54:28.400839 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:28.399850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:42173 (local address 127.0.0.1:60412)
0925 10:54:28.400417 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:28.400595 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:42173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":262,"threads_started":1}
W20260925 10:54:28.401067 1246 catalog_manager.cc:4584] TS eaefc446ad8a420cbb49ffda1e09a0be (127.31.62.129:42173): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4bdaf9ee8914453f826c1aeb8abef8e1: Network error: Client connection negotiation failed: client connection to 127.31.62.129:42173: connect: Connection refused (error 111)
Sep 25 10:54:28 dist-test-slave-nx7s krb5kdc[1216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333665, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:54:28.415932 1948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:28.400104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45710 (local address 127.31.62.130:34269)
0925 10:54:28.400633 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:28.400639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:28.400658 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:54:28.400719 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:28.400724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:28.400825 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:28.400972 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:28.402251 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:28.402995 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:28.404235 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:28.404684 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:28.404793 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:28.409561 (+ 4768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:28.409578 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:28.409595 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:28.409648 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:28.412922 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:28.413796 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:28.413805 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:28.413807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:28.413960 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:28.414433 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:28.414437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:28.414440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:28.414839 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:28.415050 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:28.415418 (+ 368us) server_negotiation.cc:299] Negotiation successful
0925 10:54:28.415636 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":287,"threads_started":1}
W20260925 10:54:28.416558 1602 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45710
W20260925 10:54:28.417160 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 1472: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:54:28.417316 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 1472
I20260925 10:54:28.435403 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 1611
Sep 25 10:54:28 dist-test-slave-nx7s krb5kdc[1216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333665, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:54:28.453130 1949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:28.436167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48404 (local address 127.31.62.131:33723)
0925 10:54:28.436689 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:28.436693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:28.436712 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:54:28.436766 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:28.436770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:28.436830 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:28.436956 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:28.438307 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:28.439067 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:28.440367 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:28.440803 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:28.440986 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:28.446137 (+ 5151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:28.446156 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:28.446175 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:28.446222 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:28.449617 (+ 3395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:28.450627 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:28.450636 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:28.450639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:28.450852 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:28.451399 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:28.451403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:28.451406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:28.451863 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:28.452092 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:28.452543 (+ 451us) server_negotiation.cc:299] Negotiation successful
0925 10:54:28.452755 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":246,"threads_started":1}
W20260925 10:54:28.454000 1741 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48404
W20260925 10:54:28.454862 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 1611: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:54:28.454993 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:28.454609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:42173 (local address 127.0.0.1:60420)
0925 10:54:28.454785 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:28.454869 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:42173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260925 10:54:28.455024 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 1611
I20260925 10:54:28.456406 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:28.455950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:42173 (local address 127.0.0.1:60426)
0925 10:54:28.456157 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:28.456240 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:42173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260925 10:54:28.476972 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 1229
I20260925 10:54:28.478228 1301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:28.595877 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:28.595398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:42173 (local address 127.0.0.1:60432)
0925 10:54:28.595577 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:28.595680 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:42173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260925 10:54:28.596657 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 1229
2026-09-25T10:54:28Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9170 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:54:29 dist-test-slave-nx7s krb5kdc[1953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:54:29 dist-test-slave-nx7s krb5kdc[1953](info): set up 2 sockets
Sep 25 10:54:29 dist-test-slave-nx7s krb5kdc[1953](info): commencing operation
krb5kdc: starting...
W20260925 10:54:30.353457 31994 mini_kdc.cc:121] Time spent starting KDC: real 1.716s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:54:30 dist-test-slave-nx7s krb5kdc[1953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:54:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:54:30.537673 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:42861
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:42861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:30.854550 1966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:30.854876 1966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:30.855001 1966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:30.862257 1966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:30.862382 1966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:30.862468 1966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:30.862538 1966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:30.862599 1966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:54:30.872610 1966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:42861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:42861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:30.873946 1966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:30.875231 1966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:30.886798 1974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:30.887013 1971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:30.886844 1972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:30.888036 1966 server_base.cc:1034] running on GCE node
I20260925 10:54:30.889022 1966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:30.890542 1966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:30.891790 1966 hybrid_clock.cc:648] HybridClock initialized: now 1790333670891733 us; error 45 us; skew 500 ppm
I20260925 10:54:30.892154 1966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:30 dist-test-slave-nx7s krb5kdc[1953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333670, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:30.895462 1966 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:54:30.897207 1966 webserver.cc:460] Webserver started at http://127.31.62.190:37091/ using document root <none> and password file <none>
I20260925 10:54:30.897933 1966 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:30.898049 1966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:30.898350 1966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:30.901454 1966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "fb461404be6f46f6a58c29ab920337c9"
format_stamp: "Formatted at 2026-09-25 10:54:30 on dist-test-slave-nx7s"
server_key: "67b187b90cb2ce67d6bf870d19c56119"
server_key_iv: "d7323ece43f0f408f183e5dfafba4b17"
server_key_version: "encryptionkey@0"
I20260925 10:54:30.902227 1966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "fb461404be6f46f6a58c29ab920337c9"
format_stamp: "Formatted at 2026-09-25 10:54:30 on dist-test-slave-nx7s"
server_key: "67b187b90cb2ce67d6bf870d19c56119"
server_key_iv: "d7323ece43f0f408f183e5dfafba4b17"
server_key_version: "encryptionkey@0"
I20260925 10:54:30.907431 1966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:54:30.910983 1981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:30.912338 1966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:54:30.912501 1966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "fb461404be6f46f6a58c29ab920337c9"
format_stamp: "Formatted at 2026-09-25 10:54:30 on dist-test-slave-nx7s"
server_key: "67b187b90cb2ce67d6bf870d19c56119"
server_key_iv: "d7323ece43f0f408f183e5dfafba4b17"
server_key_version: "encryptionkey@0"
I20260925 10:54:30.912684 1966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:30.934927 1966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:30.939577 1966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:30.939883 1966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:30.957787 1966 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:42861
I20260925 10:54:30.957793 2032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:42861 every 8 connection(s)
I20260925 10:54:30.959244 1966 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:30.963157 2033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:30.965097 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 1966
I20260925 10:54:30.965337 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:30.965937 31994 external_mini_cluster.cc:1428] Setting key 4d9bad932698e44dfc95ad2733ef4b33
I20260925 10:54:30.976025 2033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9: Bootstrap starting.
I20260925 10:54:30.979849 2033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:30.980965 2033 log.cc:824] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9: Log is configured to *not* fsync() on all Append() calls
Sep 25 10:54:30 dist-test-slave-nx7s krb5kdc[1953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:30.984479 2033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9: No bootstrap required, opened a new log
I20260925 10:54:30.991506 2036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:30.968036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53796 (local address 127.31.62.190:42861)
0925 10:54:30.968880 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:30.968899 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:30.968955 (+ 56us) server_negotiation.cc:407] Connection header received
0925 10:54:30.971218 (+ 2263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:30.971251 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:30.971815 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:30.972309 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:30.973789 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:30.975784 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:30.977278 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:30.978366 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:30.978557 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:30.982661 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:30.982672 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:30.982693 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:30.982733 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:30.986691 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:30.987468 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:30.987481 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:30.987491 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:30.987723 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:30.988058 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:30.988063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:30.988067 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:30.988818 (+ 751us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:30.989127 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:30.989518 (+ 391us) server_negotiation.cc:299] Negotiation successful
0925 10:54:30.990026 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":324,"threads_started":1}
I20260925 10:54:30.994767 2033 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb461404be6f46f6a58c29ab920337c9" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42861 } }
I20260925 10:54:30.995245 2033 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:30.995332 2033 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb461404be6f46f6a58c29ab920337c9, State: Initialized, Role: FOLLOWER
I20260925 10:54:30.995887 2033 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb461404be6f46f6a58c29ab920337c9" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42861 } }
I20260925 10:54:30.996104 2033 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:30.996209 2033 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:30.996374 2033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:30.998589 2033 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb461404be6f46f6a58c29ab920337c9" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42861 } }
I20260925 10:54:30.999083 2033 leader_election.cc:302] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb461404be6f46f6a58c29ab920337c9; no voters:
I20260925 10:54:30.999677 2033 leader_election.cc:288] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:30.999823 2038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:31.000435 2038 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [term 1 LEADER]: Becoming Leader. State: Replica: fb461404be6f46f6a58c29ab920337c9, State: Running, Role: LEADER
I20260925 10:54:31.000990 2038 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb461404be6f46f6a58c29ab920337c9" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42861 } }
I20260925 10:54:31.001964 2033 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:31.004150 2039 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb461404be6f46f6a58c29ab920337c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb461404be6f46f6a58c29ab920337c9" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42861 } } }
I20260925 10:54:31.004129 2040 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb461404be6f46f6a58c29ab920337c9. Latest consensus state: current_term: 1 leader_uuid: "fb461404be6f46f6a58c29ab920337c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb461404be6f46f6a58c29ab920337c9" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42861 } } }
I20260925 10:54:31.004559 2039 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:31.004559 2040 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:31.006742 2045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:31.012307 2045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:31.021864 2045 catalog_manager.cc:1347] Generated new cluster ID: f193ef5270a143c3b620c53238df5e33
I20260925 10:54:31.021986 2045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:31.039422 2045 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:31.040820 2045 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:31.048779 2045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb461404be6f46f6a58c29ab920337c9: Generated new TSK 0
I20260925 10:54:31.049649 2045 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:54:31.104535 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42861
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:31.455401 2059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:31.455706 2059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:31.455824 2059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:31.463141 2059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:31.463272 2059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:31.463420 2059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:54:31.473321 2059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:31.474624 2059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:31.475943 2059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:31.489404 2067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:31.489470 2065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:31.489434 2064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:31.490384 2059 server_base.cc:1034] running on GCE node
I20260925 10:54:31.491080 2059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:31.492134 2059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:31.493384 2059 hybrid_clock.cc:648] HybridClock initialized: now 1790333671493331 us; error 41 us; skew 500 ppm
I20260925 10:54:31.493772 2059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:31 dist-test-slave-nx7s krb5kdc[1953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333671, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:31.497251 2059 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:54:31.498764 2059 webserver.cc:460] Webserver started at http://127.31.62.129:34285/ using document root <none> and password file <none>
I20260925 10:54:31.499472 2059 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:31.499571 2059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:31.499869 2059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:31.502933 2059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "aa946f241640415abc91a4777e10eefe"
format_stamp: "Formatted at 2026-09-25 10:54:31 on dist-test-slave-nx7s"
server_key: "a344cea8e83ae4394165c29bc71660bd"
server_key_iv: "e6942ae6ddf005492ff5c32b873f1d21"
server_key_version: "encryptionkey@0"
I20260925 10:54:31.503764 2059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "aa946f241640415abc91a4777e10eefe"
format_stamp: "Formatted at 2026-09-25 10:54:31 on dist-test-slave-nx7s"
server_key: "a344cea8e83ae4394165c29bc71660bd"
server_key_iv: "e6942ae6ddf005492ff5c32b873f1d21"
server_key_version: "encryptionkey@0"
I20260925 10:54:31.509061 2059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 10:54:31.512539 2074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:31.513919 2059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:54:31.514082 2059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "aa946f241640415abc91a4777e10eefe"
format_stamp: "Formatted at 2026-09-25 10:54:31 on dist-test-slave-nx7s"
server_key: "a344cea8e83ae4394165c29bc71660bd"
server_key_iv: "e6942ae6ddf005492ff5c32b873f1d21"
server_key_version: "encryptionkey@0"
I20260925 10:54:31.514266 2059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:31.535851 2059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:31.540103 2059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:31.540375 2059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:31.541397 2059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:31.542721 2059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:31.542800 2059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:31.542883 2059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:31.542927 2059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:31.573611 2059 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:35415
I20260925 10:54:31.573691 2186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:35415 every 8 connection(s)
I20260925 10:54:31.574934 2059 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:54:31.575475 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 2059
I20260925 10:54:31.575663 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:54:31.576476 31994 external_mini_cluster.cc:1428] Setting key 896ee482c210ce136b4fe8b1ed3c4a97
Sep 25 10:54:31 dist-test-slave-nx7s krb5kdc[1953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333671, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
I20260925 10:54:31.597995 2189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:31.577646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:56331 (local address 127.31.62.190:42861)
0925 10:54:31.578143 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:31.578149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:31.579469 (+ 1320us) server_negotiation.cc:407] Connection header received
0925 10:54:31.582050 (+ 2581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:31.582054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:31.582145 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:31.582248 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:31.584287 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:31.585370 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:31.586995 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:31.587589 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:31.587710 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:31.591394 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:31.591401 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:31.591406 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:31.591461 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:31.594795 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:31.595601 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:31.595606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:31.595608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:31.595739 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:31.596155 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:31.596160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:31.596162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:31.596444 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:31.596586 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:31.597354 (+ 768us) server_negotiation.cc:299] Negotiation successful
0925 10:54:31.597572 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":218,"threads_started":1}
I20260925 10:54:31.598984 2190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:31.578371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:42861 (local address 127.31.62.129:56331)
0925 10:54:31.579283 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:31.579341 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:31.581698 (+ 2357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:31.582552 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:31.582564 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:31.583289 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:31.584074 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:31.584097 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:31.585551 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:31.585555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:31.586845 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:31.586857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:31.587800 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:31.587805 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:31.588018 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:31.588679 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:31.588706 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:31.591132 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:31.594964 (+ 3832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:31.594972 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:31.594987 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:31.595470 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:31.595832 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:31.595836 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:31.595838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:31.596044 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:31.596569 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:31.596577 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:31.596964 (+ 387us) client_negotiation.cc:769] Sending connection context
0925 10:54:31.597254 (+ 290us) client_negotiation.cc:240] Negotiation successful
0925 10:54:31.597805 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":343,"threads_started":1}
I20260925 10:54:31.600570 2187 heartbeater.cc:347] Connected to a master server at 127.31.62.190:42861
I20260925 10:54:31.600906 2187 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:31.601611 2187 heartbeater.cc:511] Master 127.31.62.190:42861 requested a full tablet report, sending...
I20260925 10:54:31.603653 1998 ts_manager.cc:194] Registered new tserver with Master: aa946f241640415abc91a4777e10eefe (127.31.62.129:35415)
I20260925 10:54:31.605141 1998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:56331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:54:31.620450 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42861
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:31.943698 2193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:31.944038 2193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:31.944185 2193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:31.951596 2193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:31.951720 2193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:31.951833 2193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:54:31.961699 2193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:31.963070 2193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:31.964515 2193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:31.978431 2199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:31.978488 2198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:31.978510 2201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:31.979110 2193 server_base.cc:1034] running on GCE node
I20260925 10:54:31.979822 2193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:31.980885 2193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:31.982115 2193 hybrid_clock.cc:648] HybridClock initialized: now 1790333671982050 us; error 56 us; skew 500 ppm
I20260925 10:54:31.982481 2193 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:31 dist-test-slave-nx7s krb5kdc[1953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333671, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:31.985813 2193 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:54:31.987367 2193 webserver.cc:460] Webserver started at http://127.31.62.130:35669/ using document root <none> and password file <none>
I20260925 10:54:31.988044 2193 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:31.988142 2193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:31.988421 2193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:31.991530 2193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "5b3fe1bfd25d440d82e0239430075cd7"
format_stamp: "Formatted at 2026-09-25 10:54:31 on dist-test-slave-nx7s"
server_key: "566ed31410b73e7c75196ac25744e581"
server_key_iv: "8c0d45814f1cc0727a7b222b125536e4"
server_key_version: "encryptionkey@0"
I20260925 10:54:31.992291 2193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "5b3fe1bfd25d440d82e0239430075cd7"
format_stamp: "Formatted at 2026-09-25 10:54:31 on dist-test-slave-nx7s"
server_key: "566ed31410b73e7c75196ac25744e581"
server_key_iv: "8c0d45814f1cc0727a7b222b125536e4"
server_key_version: "encryptionkey@0"
I20260925 10:54:31.997843 2193 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260925 10:54:32.001639 2208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:32.003273 2193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:54:32.003455 2193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "5b3fe1bfd25d440d82e0239430075cd7"
format_stamp: "Formatted at 2026-09-25 10:54:31 on dist-test-slave-nx7s"
server_key: "566ed31410b73e7c75196ac25744e581"
server_key_iv: "8c0d45814f1cc0727a7b222b125536e4"
server_key_version: "encryptionkey@0"
I20260925 10:54:32.003638 2193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:32.021137 2193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:32.025563 2193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:32.025846 2193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:32.026784 2193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:32.028159 2193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:32.028231 2193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:32.028331 2193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:32.028388 2193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:32.057838 2193 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:43401
I20260925 10:54:32.057873 2320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:43401 every 8 connection(s)
I20260925 10:54:32.059048 2193 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:54:32.068073 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 2193
I20260925 10:54:32.068212 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:54:32.068670 31994 external_mini_cluster.cc:1428] Setting key 7c44f93e3a9d14565f3340e87d6ecfab
Sep 25 10:54:32 dist-test-slave-nx7s krb5kdc[1953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333671, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:32.079856 2189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:32.061515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:49449 (local address 127.31.62.190:42861)
0925 10:54:32.061718 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:32.061724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:32.062937 (+ 1213us) server_negotiation.cc:407] Connection header received
0925 10:54:32.064780 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:32.064785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:32.064908 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:32.064998 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:32.066603 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:32.067328 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:32.069026 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:32.069505 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:32.069605 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:32.073662 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:32.073667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:32.073671 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:32.073712 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:32.076669 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:32.077552 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:32.077558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:32.077561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:32.077710 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:32.078161 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:32.078165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:32.078167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:32.078432 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:32.078560 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:32.079286 (+ 726us) server_negotiation.cc:299] Negotiation successful
0925 10:54:32.079517 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 10:54:32.081120 2323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:32.062056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:42861 (local address 127.31.62.130:49449)
0925 10:54:32.062775 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:32.062814 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:32.064490 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:32.065288 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:32.065298 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:32.065837 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:32.066438 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:32.066457 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:32.067480 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:32.067484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:32.068881 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:32.068897 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:32.069728 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:32.069736 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:32.069983 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:32.070673 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:32.070705 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:32.073409 (+ 2704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:32.076823 (+ 3414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:32.076843 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:32.076893 (+ 50us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:32.077410 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:32.077831 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:32.077835 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:32.077837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:32.078033 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:32.078555 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:32.078563 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:32.078929 (+ 366us) client_negotiation.cc:769] Sending connection context
0925 10:54:32.079178 (+ 249us) client_negotiation.cc:240] Negotiation successful
0925 10:54:32.079764 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":282,"threads_started":1}
I20260925 10:54:32.082931 2321 heartbeater.cc:347] Connected to a master server at 127.31.62.190:42861
I20260925 10:54:32.083243 2321 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:32.084090 2321 heartbeater.cc:511] Master 127.31.62.190:42861 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
I20260925 10:54:32.085721 1998 ts_manager.cc:194] Registered new tserver with Master: 5b3fe1bfd25d440d82e0239430075cd7 (127.31.62.130:43401)
I20260925 10:54:32.086496 1998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:49449
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:54:32.109958 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42861
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:32.446364 2326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:32.446683 2326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:32.446815 2326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:32.454147 2326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:32.454293 2326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:32.454463 2326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:54:32.464272 2326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:32.465662 2326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:32.466998 2326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:32.480341 2331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:32.480346 2332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:32.480376 2334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:32.481194 2326 server_base.cc:1034] running on GCE node
I20260925 10:54:32.481832 2326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:32.482852 2326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:32.484047 2326 hybrid_clock.cc:648] HybridClock initialized: now 1790333672483935 us; error 112 us; skew 500 ppm
I20260925 10:54:32.484387 2326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:32 dist-test-slave-nx7s krb5kdc[1953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333672, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:32.487499 2326 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:54:32.489046 2326 webserver.cc:460] Webserver started at http://127.31.62.131:43973/ using document root <none> and password file <none>
I20260925 10:54:32.489739 2326 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:32.489854 2326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:32.490141 2326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:32.493052 2326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "67146ffd61a6490a8ca7f7fb415dafb5"
format_stamp: "Formatted at 2026-09-25 10:54:32 on dist-test-slave-nx7s"
server_key: "09d4ed4edaa555c70cc485715fd1e345"
server_key_iv: "db0e1f1156f338164c0dcb33a75418a2"
server_key_version: "encryptionkey@0"
I20260925 10:54:32.493818 2326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "67146ffd61a6490a8ca7f7fb415dafb5"
format_stamp: "Formatted at 2026-09-25 10:54:32 on dist-test-slave-nx7s"
server_key: "09d4ed4edaa555c70cc485715fd1e345"
server_key_iv: "db0e1f1156f338164c0dcb33a75418a2"
server_key_version: "encryptionkey@0"
I20260925 10:54:32.499083 2326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260925 10:54:32.502686 2341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:32.504091 2326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:54:32.504244 2326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "67146ffd61a6490a8ca7f7fb415dafb5"
format_stamp: "Formatted at 2026-09-25 10:54:32 on dist-test-slave-nx7s"
server_key: "09d4ed4edaa555c70cc485715fd1e345"
server_key_iv: "db0e1f1156f338164c0dcb33a75418a2"
server_key_version: "encryptionkey@0"
I20260925 10:54:32.504431 2326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:32.531040 2326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:32.535120 2326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:32.535398 2326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:32.536319 2326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:32.537655 2326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:32.537740 2326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:32.537858 2326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:32.537909 2326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:32.568537 2326 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:43611
I20260925 10:54:32.568601 2453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:43611 every 8 connection(s)
I20260925 10:54:32.569808 2326 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:54:32.578060 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 2326
I20260925 10:54:32.578222 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:54:32.578732 31994 external_mini_cluster.cc:1428] Setting key 23fec764f08f7fed26eeaf5b75fbc96f
Sep 25 10:54:32 dist-test-slave-nx7s krb5kdc[1953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333672, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:32.591101 2189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:32.572245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:45975 (local address 127.31.62.190:42861)
0925 10:54:32.572420 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:32.572425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:32.573709 (+ 1284us) server_negotiation.cc:407] Connection header received
0925 10:54:32.576265 (+ 2556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:32.576269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:32.576342 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:32.576427 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:32.578457 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:32.579456 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:32.581297 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:32.581907 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:32.582013 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:32.585600 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:32.585605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:32.585608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:32.585642 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:32.588027 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:32.588815 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:32.588819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:32.588822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:32.589005 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:32.589475 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:32.589479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:32.589480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:32.589825 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:32.589968 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:32.590668 (+ 700us) server_negotiation.cc:299] Negotiation successful
0925 10:54:32.590840 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 10:54:32.592129 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:32.572744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:42861 (local address 127.31.62.131:45975)
0925 10:54:32.573528 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:32.573576 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:32.575918 (+ 2342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:32.576706 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:32.576716 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:32.577462 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:32.578210 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:32.578235 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:32.579594 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:32.579599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:32.581146 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:32.581159 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:32.582124 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:32.582130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:32.582336 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:32.582999 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:32.583022 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:32.585371 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:32.588170 (+ 2799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:32.588177 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:32.588243 (+ 66us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:32.588667 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:32.589112 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:32.589116 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:32.589118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:32.589375 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:32.589965 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:32.589972 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:32.590350 (+ 378us) client_negotiation.cc:769] Sending connection context
0925 10:54:32.590574 (+ 224us) client_negotiation.cc:240] Negotiation successful
0925 10:54:32.590994 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":341,"threads_started":1}
I20260925 10:54:32.593580 2454 heartbeater.cc:347] Connected to a master server at 127.31.62.190:42861
I20260925 10:54:32.593904 2454 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:32.594628 2454 heartbeater.cc:511] Master 127.31.62.190:42861 requested a full tablet report, sending...
I20260925 10:54:32.596031 1998 ts_manager.cc:194] Registered new tserver with Master: 67146ffd61a6490a8ca7f7fb415dafb5 (127.31.62.131:43611)
I20260925 10:54:32.596899 1998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:45975
I20260925 10:54:32.606527 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:54:32.608174 2187 heartbeater.cc:503] Master 127.31.62.190:42861 was elected leader, sending a full tablet report...
W20260925 10:54:32.635519 2189 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:54:32.633112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53798 (local address 127.31.62.190:42861)
0925 10:54:32.633274 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:32.633278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:32.633727 (+ 449us) server_negotiation.cc:407] Connection header received
0925 10:54:32.633903 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:32.633907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:32.633973 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:32.634050 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:32.635299 (+ 1249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53798: BlockingRecv error: recv got EOF from 127.0.0.1:53798 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 10:54:32.638533 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 2059
W20260925 10:54:32.641434 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 2059: Not authorized: Client connection negotiation failed: client connection to 127.31.62.129:35415: server requires authentication, but client does not have Kerberos credentials available
I20260925 10:54:32.641587 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 2059
I20260925 10:54:32.653973 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 2193
W20260925 10:54:32.656514 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 2193: Not authorized: Client connection negotiation failed: client connection to 127.31.62.130:43401: server requires authentication, but client does not have Kerberos credentials available
I20260925 10:54:32.656620 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 2193
I20260925 10:54:32.668632 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 2326
W20260925 10:54:32.671586 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 2326: Not authorized: Client connection negotiation failed: client connection to 127.31.62.131:43611: server requires authentication, but client does not have Kerberos credentials available
I20260925 10:54:32.671765 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 2326
I20260925 10:54:32.683537 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 1966
I20260925 10:54:32.684669 2028 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:32.785182 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 1966
2026-09-25T10:54:32Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4184 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:54:32 dist-test-slave-nx7s krb5kdc[2471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:54:32 dist-test-slave-nx7s krb5kdc[2471](info): set up 2 sockets
Sep 25 10:54:32 dist-test-slave-nx7s krb5kdc[2471](info): commencing operation
krb5kdc: starting...
W20260925 10:54:33.785553 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.963s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:54:33 dist-test-slave-nx7s krb5kdc[2471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:54:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:54:33.973742 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:42159
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:40131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:42159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:34.289532 2484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:34.289834 2484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:34.289944 2484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:34.297408 2484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:34.297531 2484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:34.297578 2484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:34.297657 2484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:34.297705 2484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:54:34.307794 2484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:40131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:42159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:42159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:34.309139 2484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:34.310427 2484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:34.321856 2492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:34.321913 2490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:34.322084 2489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:34.322909 2484 server_base.cc:1034] running on GCE node
I20260925 10:54:34.323839 2484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:34.325402 2484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:34.326635 2484 hybrid_clock.cc:648] HybridClock initialized: now 1790333674326589 us; error 28 us; skew 500 ppm
I20260925 10:54:34.327013 2484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:34 dist-test-slave-nx7s krb5kdc[2471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333674, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:34.334055 2484 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:54:34.335810 2484 webserver.cc:460] Webserver started at http://127.31.62.190:34061/ using document root <none> and password file <none>
I20260925 10:54:34.336561 2484 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:34.336687 2484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:34.337034 2484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:34.340168 2484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "09ca7c3c26594d72b662d07a41f575e2"
format_stamp: "Formatted at 2026-09-25 10:54:34 on dist-test-slave-nx7s"
server_key: "63619e88d3e978259ded4a68e2fa926e"
server_key_iv: "2136d91ede23d64fb383538dbdcf6ff1"
server_key_version: "encryptionkey@0"
I20260925 10:54:34.341079 2484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "09ca7c3c26594d72b662d07a41f575e2"
format_stamp: "Formatted at 2026-09-25 10:54:34 on dist-test-slave-nx7s"
server_key: "63619e88d3e978259ded4a68e2fa926e"
server_key_iv: "2136d91ede23d64fb383538dbdcf6ff1"
server_key_version: "encryptionkey@0"
I20260925 10:54:34.346819 2484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:54:34.350754 2499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:34.352392 2484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:54:34.352592 2484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "09ca7c3c26594d72b662d07a41f575e2"
format_stamp: "Formatted at 2026-09-25 10:54:34 on dist-test-slave-nx7s"
server_key: "63619e88d3e978259ded4a68e2fa926e"
server_key_iv: "2136d91ede23d64fb383538dbdcf6ff1"
server_key_version: "encryptionkey@0"
I20260925 10:54:34.352819 2484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:34.371765 2484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:34.376055 2484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:34.376334 2484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:34.394055 2484 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:42159
I20260925 10:54:34.394094 2550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:42159 every 8 connection(s)
I20260925 10:54:34.395448 2484 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:34.399132 2551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:34.401283 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 2484
I20260925 10:54:34.401477 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:34.402056 31994 external_mini_cluster.cc:1428] Setting key 494bb4a2f9c3520fb7c76042c8d0b844
I20260925 10:54:34.412060 2551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2: Bootstrap starting.
I20260925 10:54:34.415745 2551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:34.416749 2551 log.cc:824] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2: Log is configured to *not* fsync() on all Append() calls
Sep 25 10:54:34 dist-test-slave-nx7s krb5kdc[2471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:34.419852 2551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2: No bootstrap required, opened a new log
I20260925 10:54:34.426151 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:34.403866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51618 (local address 127.31.62.190:42159)
0925 10:54:34.404738 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:34.404754 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:34.404831 (+ 77us) server_negotiation.cc:407] Connection header received
0925 10:54:34.407175 (+ 2344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:34.407206 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:34.407720 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:34.408223 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:34.409698 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:34.411648 (+ 1950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:34.413111 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:34.414189 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:34.414361 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:34.418512 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:34.418520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:34.418537 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:34.418573 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:34.421658 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:34.422410 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:34.422420 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:34.422434 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:34.422618 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:34.423016 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:34.423019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:34.423022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:34.423684 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:34.423987 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:34.424448 (+ 461us) server_negotiation.cc:299] Negotiation successful
0925 10:54:34.424971 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":360,"threads_started":1}
I20260925 10:54:34.428102 2551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ca7c3c26594d72b662d07a41f575e2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42159 } }
I20260925 10:54:34.428574 2551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:34.428682 2551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09ca7c3c26594d72b662d07a41f575e2, State: Initialized, Role: FOLLOWER
I20260925 10:54:34.429286 2551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ca7c3c26594d72b662d07a41f575e2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42159 } }
I20260925 10:54:34.429523 2551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:34.429697 2551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:34.429869 2551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:34.430611 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:54:34.432222 2551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ca7c3c26594d72b662d07a41f575e2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42159 } }
I20260925 10:54:34.432756 2551 leader_election.cc:302] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09ca7c3c26594d72b662d07a41f575e2; no voters:
I20260925 10:54:34.433401 2551 leader_election.cc:288] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:34.433560 2556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:34.434609 2556 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [term 1 LEADER]: Becoming Leader. State: Replica: 09ca7c3c26594d72b662d07a41f575e2, State: Running, Role: LEADER
I20260925 10:54:34.435227 2556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ca7c3c26594d72b662d07a41f575e2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42159 } }
I20260925 10:54:34.435616 2551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:34.439284 2557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09ca7c3c26594d72b662d07a41f575e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ca7c3c26594d72b662d07a41f575e2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42159 } } }
I20260925 10:54:34.439370 2558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09ca7c3c26594d72b662d07a41f575e2. Latest consensus state: current_term: 1 leader_uuid: "09ca7c3c26594d72b662d07a41f575e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ca7c3c26594d72b662d07a41f575e2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42159 } } }
I20260925 10:54:34.439687 2557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:34.439730 2558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:34.451511 2563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:34.460289 2563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:34.471590 2563 catalog_manager.cc:1347] Generated new cluster ID: 9355c6cdc35c44a8a7b72137591dd1ee
I20260925 10:54:34.471720 2563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:34.485939 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:34.474158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51620 (local address 127.31.62.190:42159)
0925 10:54:34.474329 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:34.474334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:34.477488 (+ 3154us) server_negotiation.cc:407] Connection header received
0925 10:54:34.477676 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:34.477682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:34.477787 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:34.477908 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:34.479078 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:34.480194 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:34.482012 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:34.482697 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:34.482843 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:34.484037 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:34.484044 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:34.484049 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:34.485066 (+ 1017us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:34.485168 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:34.485209 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:34.485348 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:34.485477 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 10:54:34.485590 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 10:54:34.489188 2563 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:34.491117 2563 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:34.498452 2563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09ca7c3c26594d72b662d07a41f575e2: Generated new TSK 0
I20260925 10:54:34.499326 2563 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:54:34.509001 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 2484
I20260925 10:54:34.510313 2546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:34.597931 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 2484
2026-09-25T10:54:34Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1807 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:54:34 dist-test-slave-nx7s krb5kdc[2585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:54:34 dist-test-slave-nx7s krb5kdc[2585](info): set up 2 sockets
Sep 25 10:54:34 dist-test-slave-nx7s krb5kdc[2585](info): commencing operation
krb5kdc: starting...
W20260925 10:54:35.593396 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.964s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:54:35 dist-test-slave-nx7s krb5kdc[2585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:54:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:54:35.778491 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41273
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:45563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:41273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:36.102192 2598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:36.102516 2598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:36.102632 2598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:36.110301 2598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:36.110443 2598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:36.110520 2598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:36.110594 2598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:36.110656 2598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:54:36.120978 2598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:45563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:41273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:36.122371 2598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:36.123668 2598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:36.135437 2603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:36.135459 2604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:36.135414 2606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:36.136329 2598 server_base.cc:1034] running on GCE node
I20260925 10:54:36.137217 2598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:36.138726 2598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:36.139951 2598 hybrid_clock.cc:648] HybridClock initialized: now 1790333676139895 us; error 45 us; skew 500 ppm
I20260925 10:54:36.140309 2598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:36 dist-test-slave-nx7s krb5kdc[2585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333676, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:36.144164 2598 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:54:36.145910 2598 webserver.cc:460] Webserver started at http://127.31.62.190:35243/ using document root <none> and password file <none>
I20260925 10:54:36.146595 2598 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:36.146701 2598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:36.146991 2598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:36.150017 2598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "083c55a027f842a5a598f3c0bcfc4ec9"
format_stamp: "Formatted at 2026-09-25 10:54:36 on dist-test-slave-nx7s"
server_key: "83028016696bf39896102f5c3b31c473"
server_key_iv: "25821eaf5850d6b76cd0e6f1b4cbf770"
server_key_version: "encryptionkey@0"
I20260925 10:54:36.150782 2598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "083c55a027f842a5a598f3c0bcfc4ec9"
format_stamp: "Formatted at 2026-09-25 10:54:36 on dist-test-slave-nx7s"
server_key: "83028016696bf39896102f5c3b31c473"
server_key_iv: "25821eaf5850d6b76cd0e6f1b4cbf770"
server_key_version: "encryptionkey@0"
I20260925 10:54:36.156095 2598 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260925 10:54:36.159658 2613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:36.161221 2598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:54:36.161388 2598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "083c55a027f842a5a598f3c0bcfc4ec9"
format_stamp: "Formatted at 2026-09-25 10:54:36 on dist-test-slave-nx7s"
server_key: "83028016696bf39896102f5c3b31c473"
server_key_iv: "25821eaf5850d6b76cd0e6f1b4cbf770"
server_key_version: "encryptionkey@0"
I20260925 10:54:36.161581 2598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:36.181140 2598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:36.185710 2598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:36.185982 2598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:36.203864 2598 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:41273
I20260925 10:54:36.203902 2664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:41273 every 8 connection(s)
I20260925 10:54:36.205313 2598 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:36.207425 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 2598
I20260925 10:54:36.207612 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:36.208204 31994 external_mini_cluster.cc:1428] Setting key a928aa3c4341d9b2bc3a0576111bee59
I20260925 10:54:36.209455 2665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 10:54:36 dist-test-slave-nx7s krb5kdc[2585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:36.223344 2665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9: Bootstrap starting.
I20260925 10:54:36.226666 2665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:36.227782 2665 log.cc:824] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:36.230840 2665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9: No bootstrap required, opened a new log
I20260925 10:54:36.231271 2668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:36.210242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51186 (local address 127.31.62.190:41273)
0925 10:54:36.211000 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:36.211018 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:36.211068 (+ 50us) server_negotiation.cc:407] Connection header received
0925 10:54:36.212897 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:36.212928 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:36.213506 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:36.214006 (+ 500us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:36.215307 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:36.217352 (+ 2045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:36.218641 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:36.219679 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:36.219897 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:36.223687 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:36.223695 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:36.223712 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:36.223755 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:36.226678 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:36.227711 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:36.227718 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:36.227725 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:36.227976 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:36.228346 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:36.228351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:36.228353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:36.229016 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:36.229308 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:36.229740 (+ 432us) server_negotiation.cc:299] Negotiation successful
0925 10:54:36.230178 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":299,"threads_started":1}
I20260925 10:54:36.238667 2665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083c55a027f842a5a598f3c0bcfc4ec9" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41273 } }
I20260925 10:54:36.239259 2665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:36.239370 2665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083c55a027f842a5a598f3c0bcfc4ec9, State: Initialized, Role: FOLLOWER
I20260925 10:54:36.240064 2665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083c55a027f842a5a598f3c0bcfc4ec9" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41273 } }
I20260925 10:54:36.240309 2665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:36.240440 2665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:36.240602 2665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:36.242847 2665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083c55a027f842a5a598f3c0bcfc4ec9" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41273 } }
I20260925 10:54:36.243403 2665 leader_election.cc:302] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 083c55a027f842a5a598f3c0bcfc4ec9; no voters:
I20260925 10:54:36.244088 2665 leader_election.cc:288] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:36.244179 2670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:36.244837 2670 raft_consensus.cc:686] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [term 1 LEADER]: Becoming Leader. State: Replica: 083c55a027f842a5a598f3c0bcfc4ec9, State: Running, Role: LEADER
I20260925 10:54:36.245479 2670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083c55a027f842a5a598f3c0bcfc4ec9" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41273 } }
I20260925 10:54:36.246198 2665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:36.248236 2672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 083c55a027f842a5a598f3c0bcfc4ec9. Latest consensus state: current_term: 1 leader_uuid: "083c55a027f842a5a598f3c0bcfc4ec9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083c55a027f842a5a598f3c0bcfc4ec9" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41273 } } }
I20260925 10:54:36.248637 2672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:36.248453 2671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "083c55a027f842a5a598f3c0bcfc4ec9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083c55a027f842a5a598f3c0bcfc4ec9" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41273 } } }
I20260925 10:54:36.248785 2671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:36.253764 2677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:36.258661 2677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:36.267531 2677 catalog_manager.cc:1347] Generated new cluster ID: 6364ebb0272f4940b09a7b263f939c0d
I20260925 10:54:36.267653 2677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:36.292665 2677 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:36.294720 2677 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:36.318943 2677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 083c55a027f842a5a598f3c0bcfc4ec9: Generated new TSK 0
I20260925 10:54:36.319929 2677 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:54:36.378002 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41273
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:45563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:36.701795 2691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:36.702104 2691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:36.702219 2691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:36.709556 2691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:36.709700 2691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:36.709856 2691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:54:36.719825 2691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:45563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:36.721228 2691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:36.722620 2691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:36.735349 2697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:36.735414 2696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:36.735603 2699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:36.736032 2691 server_base.cc:1034] running on GCE node
I20260925 10:54:36.736653 2691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:36.737632 2691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:36.738852 2691 hybrid_clock.cc:648] HybridClock initialized: now 1790333676738791 us; error 53 us; skew 500 ppm
I20260925 10:54:36.739176 2691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:36 dist-test-slave-nx7s krb5kdc[2585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333676, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:36.742515 2691 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:54:36.744082 2691 webserver.cc:460] Webserver started at http://127.31.62.129:39257/ using document root <none> and password file <none>
I20260925 10:54:36.744771 2691 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:36.744938 2691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:36.745260 2691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:36.748298 2691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "5608b4b8fac8467e915da01c94f9361b"
format_stamp: "Formatted at 2026-09-25 10:54:36 on dist-test-slave-nx7s"
server_key: "47ae93ab79895b9af7e5db8c0186501f"
server_key_iv: "cce8d33590725a57b485e7bbc9c1880f"
server_key_version: "encryptionkey@0"
I20260925 10:54:36.749121 2691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "5608b4b8fac8467e915da01c94f9361b"
format_stamp: "Formatted at 2026-09-25 10:54:36 on dist-test-slave-nx7s"
server_key: "47ae93ab79895b9af7e5db8c0186501f"
server_key_iv: "cce8d33590725a57b485e7bbc9c1880f"
server_key_version: "encryptionkey@0"
I20260925 10:54:36.754560 2691 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 10:54:36.759147 2706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:36.761016 2691 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 10:54:36.761220 2691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "5608b4b8fac8467e915da01c94f9361b"
format_stamp: "Formatted at 2026-09-25 10:54:36 on dist-test-slave-nx7s"
server_key: "47ae93ab79895b9af7e5db8c0186501f"
server_key_iv: "cce8d33590725a57b485e7bbc9c1880f"
server_key_version: "encryptionkey@0"
I20260925 10:54:36.761435 2691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:36.794756 2691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:36.800213 2691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:36.800591 2691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:36.801960 2691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:36.804167 2691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:36.804282 2691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:36.804418 2691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:36.804484 2691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:36.836369 2691 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:33619
I20260925 10:54:36.836468 2818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:33619 every 8 connection(s)
I20260925 10:54:36.837790 2691 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:54:36.838593 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 2691
I20260925 10:54:36.838822 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:54:36.839622 31994 external_mini_cluster.cc:1428] Setting key 6d84b98153a371b0ddcff1a62bac7a35
Sep 25 10:54:36 dist-test-slave-nx7s krb5kdc[2585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333676, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
I20260925 10:54:36.861909 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:36.840964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:33927 (local address 127.31.62.190:41273)
0925 10:54:36.843544 (+ 2580us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:36.843551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:36.843575 (+ 24us) server_negotiation.cc:407] Connection header received
0925 10:54:36.845030 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:36.845036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:36.845133 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:36.845230 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:36.847066 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:36.847876 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:36.850242 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:36.850668 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:36.850777 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:36.854997 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:36.855004 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:36.855009 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:36.855053 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:36.858386 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:36.859366 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:36.859371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:36.859374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:36.859525 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:36.860001 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:36.860005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:36.860007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:36.860365 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:36.860496 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:36.861380 (+ 884us) server_negotiation.cc:299] Negotiation successful
0925 10:54:36.861555 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2454,"thread_start_us":2356,"threads_started":1}
I20260925 10:54:36.863027 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:36.841883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41273 (local address 127.31.62.129:33927)
0925 10:54:36.842775 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:36.842828 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:36.844680 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:36.845517 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:36.845527 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:36.846177 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:36.846856 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:36.846876 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:36.848061 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:36.848067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:36.850064 (+ 1997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:36.850080 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:36.850899 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:36.850906 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:36.851190 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:36.851919 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:36.851951 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:36.854676 (+ 2725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:36.858572 (+ 3896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:36.858580 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:36.858596 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:36.859110 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:36.859648 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:36.859652 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:36.859654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:36.859877 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:36.860515 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:36.860521 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:36.860989 (+ 468us) client_negotiation.cc:769] Sending connection context
0925 10:54:36.861250 (+ 261us) client_negotiation.cc:240] Negotiation successful
0925 10:54:36.861748 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":340,"threads_started":1}
I20260925 10:54:36.864784 2819 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41273
I20260925 10:54:36.865183 2819 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:36.866068 2819 heartbeater.cc:511] Master 127.31.62.190:41273 requested a full tablet report, sending...
I20260925 10:54:36.868449 2630 ts_manager.cc:194] Registered new tserver with Master: 5608b4b8fac8467e915da01c94f9361b (127.31.62.129:33619)
I20260925 10:54:36.869992 2630 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:33927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:54:36.888715 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41273
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:45563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:37.218458 2825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:37.218860 2825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:37.219005 2825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:37.228609 2825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:37.228741 2825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:37.228894 2825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:54:37.239917 2825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:45563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:37.241671 2825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:37.243057 2825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:37.256896 2833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:37.256896 2830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:37.256898 2831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:37.257540 2825 server_base.cc:1034] running on GCE node
I20260925 10:54:37.258222 2825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:37.259189 2825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:37.260390 2825 hybrid_clock.cc:648] HybridClock initialized: now 1790333677260324 us; error 59 us; skew 500 ppm
I20260925 10:54:37.260710 2825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:37 dist-test-slave-nx7s krb5kdc[2585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333677, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:37.263878 2825 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:54:37.265625 2825 webserver.cc:460] Webserver started at http://127.31.62.130:46873/ using document root <none> and password file <none>
I20260925 10:54:37.266551 2825 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:37.266681 2825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:37.266987 2825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:37.270054 2825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "2ba7f0622698486790879ec0c2983b97"
format_stamp: "Formatted at 2026-09-25 10:54:37 on dist-test-slave-nx7s"
server_key: "06c118f1a59750a98a19632b3d872a4f"
server_key_iv: "f541fec66d57980f074d3f3b2765cd95"
server_key_version: "encryptionkey@0"
I20260925 10:54:37.270833 2825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "2ba7f0622698486790879ec0c2983b97"
format_stamp: "Formatted at 2026-09-25 10:54:37 on dist-test-slave-nx7s"
server_key: "06c118f1a59750a98a19632b3d872a4f"
server_key_iv: "f541fec66d57980f074d3f3b2765cd95"
server_key_version: "encryptionkey@0"
I20260925 10:54:37.276237 2825 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 10:54:37.280088 2840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:37.281792 2825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:54:37.281982 2825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "2ba7f0622698486790879ec0c2983b97"
format_stamp: "Formatted at 2026-09-25 10:54:37 on dist-test-slave-nx7s"
server_key: "06c118f1a59750a98a19632b3d872a4f"
server_key_iv: "f541fec66d57980f074d3f3b2765cd95"
server_key_version: "encryptionkey@0"
I20260925 10:54:37.282188 2825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:37.295257 2825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:37.299588 2825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:37.300086 2825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:37.301247 2825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:37.302834 2825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:37.302922 2825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:37.303053 2825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:37.303110 2825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:37.335602 2825 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:37537
I20260925 10:54:37.335636 2952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:37537 every 8 connection(s)
I20260925 10:54:37.336961 2825 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:54:37.347124 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 2825
I20260925 10:54:37.347313 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:54:37.347993 31994 external_mini_cluster.cc:1428] Setting key 2ceb32db8fbd7a83a033490117ad0065
Sep 25 10:54:37 dist-test-slave-nx7s krb5kdc[2585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333677, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:37.358983 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:37.339408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:41511 (local address 127.31.62.190:41273)
0925 10:54:37.339576 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:37.339580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:37.340807 (+ 1227us) server_negotiation.cc:407] Connection header received
0925 10:54:37.342599 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:37.342605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:37.342682 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:37.342766 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:37.344349 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.345122 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.346711 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.347150 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.347261 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:37.351433 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:37.351440 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:37.351445 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:37.351489 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:37.354903 (+ 3414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:37.356041 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:37.356047 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:37.356051 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:37.356236 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:37.356829 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:37.356835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:37.356838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:37.357280 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:37.357411 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:37.358386 (+ 975us) server_negotiation.cc:299] Negotiation successful
0925 10:54:37.358647 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 10:54:37.360769 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:37.339956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41273 (local address 127.31.62.130:41511)
0925 10:54:37.340646 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:37.340688 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:37.342310 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:37.343013 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:37.343021 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:37.343589 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:37.344177 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:37.344195 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.345259 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.345263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:37.346570 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:37.346588 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.347529 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.347537 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:37.347873 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:37.348662 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:37.348692 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:37.351168 (+ 2476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:37.355111 (+ 3943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:37.355124 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:37.355146 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:37.355871 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:37.356402 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:37.356407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:37.356411 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:37.356691 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:37.357532 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:37.357541 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:37.357933 (+ 392us) client_negotiation.cc:769] Sending connection context
0925 10:54:37.358918 (+ 985us) client_negotiation.cc:240] Negotiation successful
0925 10:54:37.359541 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":270,"threads_started":1}
I20260925 10:54:37.362533 2953 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41273
I20260925 10:54:37.362865 2953 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:37.363591 2953 heartbeater.cc:511] Master 127.31.62.190:41273 requested a full tablet report, sending...
I20260925 10:54:37.365197 2630 ts_manager.cc:194] Registered new tserver with Master: 2ba7f0622698486790879ec0c2983b97 (127.31.62.130:37537)
I20260925 10:54:37.366146 2630 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:41511
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:54:37.395970 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41273
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:45563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:37.715991 2958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:37.716295 2958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:37.716413 2958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:37.723716 2958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:37.723824 2958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:37.723971 2958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:54:37.734028 2958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:45563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:37.735423 2958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:37.736788 2958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:37.749375 2964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:37.749879 2963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:37.749933 2966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:37.750380 2958 server_base.cc:1034] running on GCE node
I20260925 10:54:37.750988 2958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:37.751921 2958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:37.753120 2958 hybrid_clock.cc:648] HybridClock initialized: now 1790333677753059 us; error 46 us; skew 500 ppm
I20260925 10:54:37.753492 2958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:37 dist-test-slave-nx7s krb5kdc[2585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333677, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:37.756449 2958 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:54:37.757843 2958 webserver.cc:460] Webserver started at http://127.31.62.131:38985/ using document root <none> and password file <none>
I20260925 10:54:37.758482 2958 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:37.758579 2958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:37.758834 2958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:37.761667 2958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "baca1c0386cb46839054e1b64df19644"
format_stamp: "Formatted at 2026-09-25 10:54:37 on dist-test-slave-nx7s"
server_key: "9f0ea5d805d5431edd8748989d865ac8"
server_key_iv: "824dcdbd182c17081e8914eea13b0f49"
server_key_version: "encryptionkey@0"
I20260925 10:54:37.762389 2958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "baca1c0386cb46839054e1b64df19644"
format_stamp: "Formatted at 2026-09-25 10:54:37 on dist-test-slave-nx7s"
server_key: "9f0ea5d805d5431edd8748989d865ac8"
server_key_iv: "824dcdbd182c17081e8914eea13b0f49"
server_key_version: "encryptionkey@0"
I20260925 10:54:37.768365 2958 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260925 10:54:37.772343 2973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:37.774067 2958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:54:37.774225 2958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "baca1c0386cb46839054e1b64df19644"
format_stamp: "Formatted at 2026-09-25 10:54:37 on dist-test-slave-nx7s"
server_key: "9f0ea5d805d5431edd8748989d865ac8"
server_key_iv: "824dcdbd182c17081e8914eea13b0f49"
server_key_version: "encryptionkey@0"
I20260925 10:54:37.774420 2958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:37.785591 2958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:37.789968 2958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:37.790220 2958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:37.791138 2958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:37.792505 2958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:37.792578 2958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:37.792670 2958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:37.792711 2958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:37.822144 2958 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:44879
I20260925 10:54:37.822211 3085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:44879 every 8 connection(s)
I20260925 10:54:37.823436 2958 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:54:37.823938 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 2958
I20260925 10:54:37.824115 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:54:37.824649 31994 external_mini_cluster.cc:1428] Setting key b5248ff22fff6934f7ad62b2b7ac70e2
Sep 25 10:54:37 dist-test-slave-nx7s krb5kdc[2585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333677, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:37.844368 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:37.825980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:53981 (local address 127.31.62.190:41273)
0925 10:54:37.826157 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:37.826161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:37.827569 (+ 1408us) server_negotiation.cc:407] Connection header received
0925 10:54:37.829494 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:37.829499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:37.829578 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:37.829658 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:37.831257 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.831993 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.833592 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.834022 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.834132 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:37.838238 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:37.838245 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:37.838250 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:37.838294 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:37.841386 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:37.842176 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:37.842242 (+ 66us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:37.842245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:37.842389 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:37.842759 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:37.842762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:37.842764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:37.843035 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:37.843139 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:37.843868 (+ 729us) server_negotiation.cc:299] Negotiation successful
0925 10:54:37.844075 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 10:54:37.845248 3088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:37.826638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41273 (local address 127.31.62.131:53981)
0925 10:54:37.827433 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:37.827468 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:37.829238 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:37.829909 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:37.829918 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:37.830445 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:37.831091 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:37.831110 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.832138 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.832141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:37.833457 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:37.833468 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.834258 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.834353 (+ 95us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:37.834571 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:37.835201 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:37.835224 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:37.837901 (+ 2677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:37.841526 (+ 3625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:37.841536 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:37.841553 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:37.842036 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:37.842483 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:37.842486 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:37.842488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:37.842677 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:37.843146 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:37.843153 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:37.843522 (+ 369us) client_negotiation.cc:769] Sending connection context
0925 10:54:37.843742 (+ 220us) client_negotiation.cc:240] Negotiation successful
0925 10:54:37.844143 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":309,"threads_started":1}
I20260925 10:54:37.846580 3086 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41273
I20260925 10:54:37.846843 3086 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:37.847544 3086 heartbeater.cc:511] Master 127.31.62.190:41273 requested a full tablet report, sending...
I20260925 10:54:37.848960 2630 ts_manager.cc:194] Registered new tserver with Master: baca1c0386cb46839054e1b64df19644 (127.31.62.131:44879)
I20260925 10:54:37.849763 2630 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:53981
I20260925 10:54:37.853505 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:54:37.873785 2819 heartbeater.cc:503] Master 127.31.62.190:41273 was elected leader, sending a full tablet report...
Sep 25 10:54:37 dist-test-slave-nx7s krb5kdc[2585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333677, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 25 10:54:37 dist-test-slave-nx7s krb5kdc[2585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333677, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:37.901437 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:37.887096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51190 (local address 127.31.62.190:41273)
0925 10:54:37.887359 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:37.887364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:37.887878 (+ 514us) server_negotiation.cc:407] Connection header received
0925 10:54:37.888040 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:37.888044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:37.888110 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:37.888241 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:37.889617 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.890336 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.891611 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.892025 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.892122 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:37.896003 (+ 3881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:37.896008 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:37.896011 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:37.896048 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:37.898784 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:37.899564 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:37.899568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:37.899570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:37.899707 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:37.900079 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:37.900083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:37.900085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:37.900403 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:37.900599 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:37.900947 (+ 348us) server_negotiation.cc:299] Negotiation successful
0925 10:54:37.901176 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
W20260925 10:54:37.902141 2630 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51190
I20260925 10:54:37.903990 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 2691
Sep 25 10:54:37 dist-test-slave-nx7s krb5kdc[2585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333677, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:54:37.919414 3097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:37.904897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42912 (local address 127.31.62.129:33619)
0925 10:54:37.905423 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:37.905430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:37.905472 (+ 42us) server_negotiation.cc:407] Connection header received
0925 10:54:37.905532 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:37.905538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:37.905756 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:37.905876 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:37.907046 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.908073 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.909553 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.910163 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.910278 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:37.914137 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:37.914149 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:37.914165 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:37.914210 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:37.916960 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:37.917699 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:37.917706 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:37.917709 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:37.917834 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:37.918125 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:37.918129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:37.918132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:37.918455 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:37.918675 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:37.918876 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 10:54:37.919068 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":244,"threads_started":1}
W20260925 10:54:37.920701 2814 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42912
W20260925 10:54:37.921819 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 2691: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:54:37.921996 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 2691
I20260925 10:54:37.934401 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 2825
Sep 25 10:54:37 dist-test-slave-nx7s krb5kdc[2585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333677, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:54:37.950474 3098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:37.935328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36986 (local address 127.31.62.130:37537)
0925 10:54:37.935887 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:37.935894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:37.935921 (+ 27us) server_negotiation.cc:407] Connection header received
0925 10:54:37.935978 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:37.935984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:37.936199 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:37.936335 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:37.937601 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.938683 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.939913 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.940481 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.940593 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:37.944975 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:37.944987 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:37.945002 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:37.945038 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:37.947548 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:37.948181 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:37.948190 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:37.948192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:37.948305 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:37.948650 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:37.948655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:37.948658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:37.949204 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:37.949484 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:37.949874 (+ 390us) server_negotiation.cc:299] Negotiation successful
0925 10:54:37.950097 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":289,"threads_started":1}
W20260925 10:54:37.951737 2948 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36986
W20260925 10:54:37.952929 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 2825: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:54:37.953090 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 2825
I20260925 10:54:37.965039 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 2958
Sep 25 10:54:37 dist-test-slave-nx7s krb5kdc[2585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333677, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:54:37.981899 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:37.966289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.31.62.131:44879)
0925 10:54:37.966915 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:37.966922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:37.966951 (+ 29us) server_negotiation.cc:407] Connection header received
0925 10:54:37.967009 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:37.967015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:37.967225 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:37.967356 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:37.968693 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.969821 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.971099 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:37.971683 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:37.971790 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:37.976086 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:37.976100 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:37.976115 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:37.976157 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:37.979114 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:37.979859 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:37.979866 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:37.979869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:37.980001 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:37.980378 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:37.980382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:37.980384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:37.980778 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:37.981063 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:37.981326 (+ 263us) server_negotiation.cc:299] Negotiation successful
0925 10:54:37.981554 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":317,"threads_started":1}
W20260925 10:54:37.983170 3081 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42760
W20260925 10:54:37.984386 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 2958: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:54:37.984536 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 2958
I20260925 10:54:37.996599 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 2598
I20260925 10:54:37.997799 2660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:38.099615 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 2598
2026-09-25T10:54:38Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3506 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:54:38 dist-test-slave-nx7s krb5kdc[3103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:54:38 dist-test-slave-nx7s krb5kdc[3103](info): set up 2 sockets
Sep 25 10:54:38 dist-test-slave-nx7s krb5kdc[3103](info): commencing operation
krb5kdc: starting...
W20260925 10:54:39.101516 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.964s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:54:39 dist-test-slave-nx7s krb5kdc[3103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:54:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:54:39.294041 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41413
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:41413
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:39.618253 3116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:39.618544 3116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:39.618630 3116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:39.625905 3116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:39.626016 3116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:39.626067 3116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:39.626123 3116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:39.626165 3116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:54:39.636197 3116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:41413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:39.637487 3116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:39.638712 3116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:39.650270 3121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:39.650345 3122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:39.650240 3124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:39.651067 3116 server_base.cc:1034] running on GCE node
I20260925 10:54:39.651852 3116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:39.653347 3116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:39.654557 3116 hybrid_clock.cc:648] HybridClock initialized: now 1790333679654500 us; error 49 us; skew 500 ppm
I20260925 10:54:39.654896 3116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:39 dist-test-slave-nx7s krb5kdc[3103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333679, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:39.664149 3116 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:54:39.665786 3116 webserver.cc:460] Webserver started at http://127.31.62.190:37787/ using document root <none> and password file <none>
I20260925 10:54:39.666513 3116 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:39.666636 3116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:39.666939 3116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:39.670111 3116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "5faa7b62961b4c57a9eed67f29b466fa"
format_stamp: "Formatted at 2026-09-25 10:54:39 on dist-test-slave-nx7s"
server_key: "f19ec69c68f26a239a45aa353700f635"
server_key_iv: "6b1f529e44e975960c34b1392f4fcd47"
server_key_version: "encryptionkey@0"
I20260925 10:54:39.670939 3116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "5faa7b62961b4c57a9eed67f29b466fa"
format_stamp: "Formatted at 2026-09-25 10:54:39 on dist-test-slave-nx7s"
server_key: "f19ec69c68f26a239a45aa353700f635"
server_key_iv: "6b1f529e44e975960c34b1392f4fcd47"
server_key_version: "encryptionkey@0"
I20260925 10:54:39.676353 3116 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 10:54:39.679965 3131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:39.681581 3116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:54:39.681737 3116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "5faa7b62961b4c57a9eed67f29b466fa"
format_stamp: "Formatted at 2026-09-25 10:54:39 on dist-test-slave-nx7s"
server_key: "f19ec69c68f26a239a45aa353700f635"
server_key_iv: "6b1f529e44e975960c34b1392f4fcd47"
server_key_version: "encryptionkey@0"
I20260925 10:54:39.681923 3116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:39.696169 3116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:39.700343 3116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:39.700610 3116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:39.717736 3116 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:41413
I20260925 10:54:39.717770 3182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:41413 every 8 connection(s)
I20260925 10:54:39.719089 3116 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:39.722414 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 3116
I20260925 10:54:39.722594 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:39.722856 3183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:39.723255 31994 external_mini_cluster.cc:1428] Setting key dbb4ecb642d84009b06f801f1d2adc1f
I20260925 10:54:39.737706 3183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa: Bootstrap starting.
Sep 25 10:54:39 dist-test-slave-nx7s krb5kdc[3103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:39.741688 3183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:39.743067 3183 log.cc:824] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:39.746733 3183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa: No bootstrap required, opened a new log
I20260925 10:54:39.748086 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:39.725158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46202 (local address 127.31.62.190:41413)
0925 10:54:39.725991 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:39.726008 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:39.726064 (+ 56us) server_negotiation.cc:407] Connection header received
0925 10:54:39.727733 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:39.727764 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:39.728340 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:39.728891 (+ 551us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:39.730257 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:39.732390 (+ 2133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:39.733750 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:39.734900 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:39.735105 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:39.739124 (+ 4019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:39.739136 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:39.739163 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:39.739228 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:39.743023 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:39.744031 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:39.744042 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:39.744049 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:39.744226 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:39.744723 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:39.744728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:39.744731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:39.745514 (+ 783us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:39.745798 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:39.746259 (+ 461us) server_negotiation.cc:299] Negotiation successful
0925 10:54:39.746704 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":336,"threads_started":1}
I20260925 10:54:39.754961 3183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5faa7b62961b4c57a9eed67f29b466fa" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41413 } }
I20260925 10:54:39.755364 3183 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:39.755465 3183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5faa7b62961b4c57a9eed67f29b466fa, State: Initialized, Role: FOLLOWER
I20260925 10:54:39.755993 3183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5faa7b62961b4c57a9eed67f29b466fa" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41413 } }
I20260925 10:54:39.756192 3183 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:39.756279 3183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:39.756402 3183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:39.758496 3183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5faa7b62961b4c57a9eed67f29b466fa" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41413 } }
I20260925 10:54:39.758972 3183 leader_election.cc:302] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5faa7b62961b4c57a9eed67f29b466fa; no voters:
I20260925 10:54:39.759528 3183 leader_election.cc:288] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:39.759656 3188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:39.760890 3188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [term 1 LEADER]: Becoming Leader. State: Replica: 5faa7b62961b4c57a9eed67f29b466fa, State: Running, Role: LEADER
I20260925 10:54:39.761457 3188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5faa7b62961b4c57a9eed67f29b466fa" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41413 } }
I20260925 10:54:39.761847 3183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:39.764287 3189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5faa7b62961b4c57a9eed67f29b466fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5faa7b62961b4c57a9eed67f29b466fa" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41413 } } }
I20260925 10:54:39.764325 3190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5faa7b62961b4c57a9eed67f29b466fa. Latest consensus state: current_term: 1 leader_uuid: "5faa7b62961b4c57a9eed67f29b466fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5faa7b62961b4c57a9eed67f29b466fa" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41413 } } }
I20260925 10:54:39.764650 3189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:39.764650 3190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:39.766426 3196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:39.771867 3196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:39.781877 3196 catalog_manager.cc:1347] Generated new cluster ID: a37fd36480da4a158af9e68c0e6623f4
I20260925 10:54:39.782015 3196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:39.796942 3196 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:39.798897 3196 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:39.813163 3196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5faa7b62961b4c57a9eed67f29b466fa: Generated new TSK 0
I20260925 10:54:39.814320 3196 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:54:39.872547 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41413
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:40.203905 3209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:40.204226 3209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:40.204371 3209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:40.211807 3209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:40.211941 3209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:40.212087 3209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:54:40.222124 3209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:40.223505 3209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:40.224884 3209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:40.238224 3214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:40.238165 3217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:40.238222 3215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:40.238981 3209 server_base.cc:1034] running on GCE node
I20260925 10:54:40.239588 3209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:40.240609 3209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:40.241842 3209 hybrid_clock.cc:648] HybridClock initialized: now 1790333680241757 us; error 71 us; skew 500 ppm
I20260925 10:54:40.242192 3209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:40 dist-test-slave-nx7s krb5kdc[3103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333680, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:40.245461 3209 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:54:40.247056 3209 webserver.cc:460] Webserver started at http://127.31.62.129:40019/ using document root <none> and password file <none>
I20260925 10:54:40.247778 3209 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:40.247892 3209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:40.248193 3209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:40.251296 3209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "4c4346f91ce14a43a8510795e851edcf"
format_stamp: "Formatted at 2026-09-25 10:54:40 on dist-test-slave-nx7s"
server_key: "4e3bc8501c48693312b2b404030c2b92"
server_key_iv: "f62f1b0acc6954799070d3b8f062073a"
server_key_version: "encryptionkey@0"
I20260925 10:54:40.252095 3209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "4c4346f91ce14a43a8510795e851edcf"
format_stamp: "Formatted at 2026-09-25 10:54:40 on dist-test-slave-nx7s"
server_key: "4e3bc8501c48693312b2b404030c2b92"
server_key_iv: "f62f1b0acc6954799070d3b8f062073a"
server_key_version: "encryptionkey@0"
I20260925 10:54:40.257462 3209 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 10:54:40.261189 3224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:40.262776 3209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:54:40.262951 3209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "4c4346f91ce14a43a8510795e851edcf"
format_stamp: "Formatted at 2026-09-25 10:54:40 on dist-test-slave-nx7s"
server_key: "4e3bc8501c48693312b2b404030c2b92"
server_key_iv: "f62f1b0acc6954799070d3b8f062073a"
server_key_version: "encryptionkey@0"
I20260925 10:54:40.263163 3209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:40.276000 3209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:40.280646 3209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:40.281006 3209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:40.282092 3209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:40.283562 3209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:40.283651 3209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:40.283788 3209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:40.283849 3209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:40.316969 3209 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:40413
I20260925 10:54:40.317029 3336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:40413 every 8 connection(s)
I20260925 10:54:40.318328 3209 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:54:40.323052 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 3209
I20260925 10:54:40.323242 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:54:40.324018 31994 external_mini_cluster.cc:1428] Setting key 6411e27a3662431938989e2e292601b8
Sep 25 10:54:40 dist-test-slave-nx7s krb5kdc[3103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333680, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:40.339363 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:40.320931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:43487 (local address 127.31.62.190:41413)
0925 10:54:40.321374 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:40.321378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:40.322299 (+ 921us) server_negotiation.cc:407] Connection header received
0925 10:54:40.324678 (+ 2379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:40.324684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:40.324779 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:40.324943 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:40.326782 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:40.327482 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:40.329591 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:40.330054 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:40.330219 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:40.333630 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:40.333635 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:40.333638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:40.333669 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:40.336332 (+ 2663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:40.337098 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:40.337104 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:40.337108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:40.337226 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:40.337643 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:40.337649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:40.337652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:40.337936 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:40.338030 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:40.338811 (+ 781us) server_negotiation.cc:299] Negotiation successful
0925 10:54:40.339022 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":199,"threads_started":1}
I20260925 10:54:40.340343 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:40.321421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41413 (local address 127.31.62.129:43487)
0925 10:54:40.322101 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:40.322142 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:40.324339 (+ 2197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:40.325241 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:40.325250 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:40.325821 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:40.326606 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:40.326631 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:40.327607 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:40.327613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:40.329391 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:40.329418 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:40.330330 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:40.330337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:40.330596 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:40.331281 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:40.331309 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:40.333411 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:40.336459 (+ 3048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:40.336469 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:40.336487 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:40.336981 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:40.337323 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:40.337328 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:40.337332 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:40.337516 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:40.338104 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:40.338110 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:40.338449 (+ 339us) client_negotiation.cc:769] Sending connection context
0925 10:54:40.338679 (+ 230us) client_negotiation.cc:240] Negotiation successful
0925 10:54:40.339208 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":261,"threads_started":1}
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
I20260925 10:54:40.341939 3337 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41413
I20260925 10:54:40.342332 3337 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:40.343044 3337 heartbeater.cc:511] Master 127.31.62.190:41413 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
I20260925 10:54:40.345268 3148 ts_manager.cc:194] Registered new tserver with Master: 4c4346f91ce14a43a8510795e851edcf (127.31.62.129:40413)
I20260925 10:54:40.346705 3148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:43487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:54:40.363961 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41413
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:40.692911 3343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:40.693216 3343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:40.693418 3343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:40.700773 3343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:40.700917 3343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:40.701076 3343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:54:40.711052 3343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:40.712387 3343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:40.713755 3343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:40.727828 3349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:40.728020 3351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:40.728119 3348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:40.728603 3343 server_base.cc:1034] running on GCE node
I20260925 10:54:40.729242 3343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:40.730314 3343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:40.731581 3343 hybrid_clock.cc:648] HybridClock initialized: now 1790333680731501 us; error 88 us; skew 500 ppm
I20260925 10:54:40.731901 3343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:40 dist-test-slave-nx7s krb5kdc[3103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333680, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:40.735020 3343 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:54:40.736474 3343 webserver.cc:460] Webserver started at http://127.31.62.130:45987/ using document root <none> and password file <none>
I20260925 10:54:40.737289 3343 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:40.737399 3343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:40.737692 3343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:40.740636 3343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "e86a2550e00746538c4c42336b7c820d"
format_stamp: "Formatted at 2026-09-25 10:54:40 on dist-test-slave-nx7s"
server_key: "483d224ca3830acc98c420a7528a4542"
server_key_iv: "d54d1ff254a93b05c97c29c112caf864"
server_key_version: "encryptionkey@0"
I20260925 10:54:40.741496 3343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "e86a2550e00746538c4c42336b7c820d"
format_stamp: "Formatted at 2026-09-25 10:54:40 on dist-test-slave-nx7s"
server_key: "483d224ca3830acc98c420a7528a4542"
server_key_iv: "d54d1ff254a93b05c97c29c112caf864"
server_key_version: "encryptionkey@0"
I20260925 10:54:40.746779 3343 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:54:40.750356 3358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:40.751856 3343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:54:40.752038 3343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "e86a2550e00746538c4c42336b7c820d"
format_stamp: "Formatted at 2026-09-25 10:54:40 on dist-test-slave-nx7s"
server_key: "483d224ca3830acc98c420a7528a4542"
server_key_iv: "d54d1ff254a93b05c97c29c112caf864"
server_key_version: "encryptionkey@0"
I20260925 10:54:40.752298 3343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:40.774973 3343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:40.779417 3343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:40.779703 3343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:40.780704 3343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:40.782400 3343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:40.782488 3343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:40.782588 3343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:40.782644 3343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:40.813717 3343 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:34819
I20260925 10:54:40.813789 3470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:34819 every 8 connection(s)
I20260925 10:54:40.814988 3343 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:54:40.822597 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 3343
I20260925 10:54:40.822762 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:54:40.823166 31994 external_mini_cluster.cc:1428] Setting key 6217086689a920e6b2ee0a8d78a06f68
Sep 25 10:54:40 dist-test-slave-nx7s krb5kdc[3103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333680, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:40.835435 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:40.817727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:34637 (local address 127.31.62.190:41413)
0925 10:54:40.817890 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:40.817895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:40.819151 (+ 1256us) server_negotiation.cc:407] Connection header received
0925 10:54:40.821269 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:40.821275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:40.821368 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:40.821456 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:40.823148 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:40.823978 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:40.825655 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:40.826073 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:40.826185 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:40.829858 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:40.829863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:40.829866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:40.829900 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:40.832282 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:40.833074 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:40.833078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:40.833081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:40.833209 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:40.833622 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:40.833625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:40.833627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:40.833909 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:40.834037 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:40.834784 (+ 747us) server_negotiation.cc:299] Negotiation successful
0925 10:54:40.835022 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 10:54:40.836535 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:40.818206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41413 (local address 127.31.62.130:34637)
0925 10:54:40.818966 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:40.819012 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:40.820956 (+ 1944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:40.821719 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:40.821729 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:40.822295 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:40.822961 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:40.822987 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:40.824130 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:40.824134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:40.825513 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:40.825525 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:40.826305 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:40.826312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:40.826530 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:40.827239 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:40.827266 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:40.829581 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:40.832429 (+ 2848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:40.832439 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:40.832460 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:40.832949 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:40.833313 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:40.833318 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:40.833322 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:40.833508 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:40.834029 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:40.834037 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:40.834409 (+ 372us) client_negotiation.cc:769] Sending connection context
0925 10:54:40.834649 (+ 240us) client_negotiation.cc:240] Negotiation successful
0925 10:54:40.835272 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":295,"threads_started":1}
I20260925 10:54:40.838059 3471 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41413
I20260925 10:54:40.838433 3471 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:40.839375 3471 heartbeater.cc:511] Master 127.31.62.190:41413 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
I20260925 10:54:40.841010 3148 ts_manager.cc:194] Registered new tserver with Master: e86a2550e00746538c4c42336b7c820d (127.31.62.130:34819)
I20260925 10:54:40.841823 3148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:34637
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:54:40.865360 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41413
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:41.187955 3476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:41.188247 3476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:41.188364 3476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:41.195557 3476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:41.195676 3476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:41.195824 3476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:54:41.205520 3476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:41.206806 3476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:41.208123 3476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:41.221330 3484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:41.221467 3482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:41.221472 3481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:41.222553 3476 server_base.cc:1034] running on GCE node
I20260925 10:54:41.223309 3476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:41.224275 3476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:41.225509 3476 hybrid_clock.cc:648] HybridClock initialized: now 1790333681225456 us; error 46 us; skew 500 ppm
I20260925 10:54:41.225852 3476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:41 dist-test-slave-nx7s krb5kdc[3103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333681, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:41.229117 3476 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:54:41.230667 3476 webserver.cc:460] Webserver started at http://127.31.62.131:33373/ using document root <none> and password file <none>
I20260925 10:54:41.231385 3476 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:41.231499 3476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:41.231793 3476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:41.234933 3476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "576e17b51b184436a1d498bfd3222e2b"
format_stamp: "Formatted at 2026-09-25 10:54:41 on dist-test-slave-nx7s"
server_key: "f3c9381a2e4934c7318ff1b71c125bc5"
server_key_iv: "b4a98f16ffa26e7aec21c76b5ee9ad7c"
server_key_version: "encryptionkey@0"
I20260925 10:54:41.235765 3476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "576e17b51b184436a1d498bfd3222e2b"
format_stamp: "Formatted at 2026-09-25 10:54:41 on dist-test-slave-nx7s"
server_key: "f3c9381a2e4934c7318ff1b71c125bc5"
server_key_iv: "b4a98f16ffa26e7aec21c76b5ee9ad7c"
server_key_version: "encryptionkey@0"
I20260925 10:54:41.241397 3476 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260925 10:54:41.245190 3491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:41.246712 3476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:54:41.246873 3476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "576e17b51b184436a1d498bfd3222e2b"
format_stamp: "Formatted at 2026-09-25 10:54:41 on dist-test-slave-nx7s"
server_key: "f3c9381a2e4934c7318ff1b71c125bc5"
server_key_iv: "b4a98f16ffa26e7aec21c76b5ee9ad7c"
server_key_version: "encryptionkey@0"
I20260925 10:54:41.247066 3476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:41.273175 3476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:41.277730 3476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:41.278014 3476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:41.279117 3476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:41.280534 3476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:41.280617 3476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:41.280745 3476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:41.280800 3476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:41.311288 3476 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:43375
I20260925 10:54:41.311358 3603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:43375 every 8 connection(s)
I20260925 10:54:41.312534 3476 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:54:41.313694 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 3476
I20260925 10:54:41.313862 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:54:41.314390 31994 external_mini_cluster.cc:1428] Setting key d9e3123004631eed1ba5db9d363871ef
Sep 25 10:54:41 dist-test-slave-nx7s krb5kdc[3103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333681, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:41.332765 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:41.315233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:54277 (local address 127.31.62.190:41413)
0925 10:54:41.315431 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:41.315437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:41.316786 (+ 1349us) server_negotiation.cc:407] Connection header received
0925 10:54:41.319350 (+ 2564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:41.319354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:41.319424 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:41.319519 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:41.321504 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:41.322394 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:41.323886 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:41.324280 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:41.324372 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:41.327572 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:41.327577 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:41.327580 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:41.327613 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:41.329985 (+ 2372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:41.330665 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:41.330668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:41.330671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:41.330807 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:41.331159 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:41.331162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:41.331164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:41.331404 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:41.331510 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:41.332216 (+ 706us) server_negotiation.cc:299] Negotiation successful
0925 10:54:41.332467 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 10:54:41.333933 3606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:41.315828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41413 (local address 127.31.62.131:54277)
0925 10:54:41.316615 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:41.316673 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:41.319025 (+ 2352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:41.319785 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:41.319797 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:41.320524 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:41.321322 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:41.321344 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:41.322544 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:41.322550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:41.323763 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:41.323775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:41.324452 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:41.324457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:41.324643 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:41.325256 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:41.325280 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:41.327366 (+ 2086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:41.330117 (+ 2751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:41.330124 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:41.330139 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:41.330567 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:41.330893 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:41.330896 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:41.330898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:41.331080 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:41.331514 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:41.331521 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:41.331846 (+ 325us) client_negotiation.cc:769] Sending connection context
0925 10:54:41.332077 (+ 231us) client_negotiation.cc:240] Negotiation successful
0925 10:54:41.332580 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":287,"threads_started":1}
I20260925 10:54:41.335126 3604 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41413
I20260925 10:54:41.335427 3604 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:41.336114 3604 heartbeater.cc:511] Master 127.31.62.190:41413 requested a full tablet report, sending...
I20260925 10:54:41.337460 3148 ts_manager.cc:194] Registered new tserver with Master: 576e17b51b184436a1d498bfd3222e2b (127.31.62.131:43375)
I20260925 10:54:41.338310 3148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:54277
I20260925 10:54:41.346771 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:54:41.349777 3337 heartbeater.cc:503] Master 127.31.62.190:41413 was elected leader, sending a full tablet report...
Sep 25 10:54:41 dist-test-slave-nx7s krb5kdc[3103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 10:54:41 dist-test-slave-nx7s krb5kdc[3103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:54:41.396068 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:41.381320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48542 (local address 127.31.62.129:40413)
0925 10:54:41.381912 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:41.381920 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:41.381948 (+ 28us) server_negotiation.cc:407] Connection header received
0925 10:54:41.382016 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:41.382022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:41.382302 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:41.382435 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:41.383917 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:41.385077 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:41.386371 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:41.387018 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:41.387122 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:41.390783 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:41.390796 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:41.390811 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:41.390851 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:41.393621 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:41.394302 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:41.394309 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:41.394311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:41.394437 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:41.394723 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:41.394726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:41.394728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:41.395055 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:41.395284 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:41.395507 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 10:54:41.395706 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":294,"threads_started":1}
Sep 25 10:54:41 dist-test-slave-nx7s krb5kdc[3103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:41.414130 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:41.400247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46212 (local address 127.31.62.190:41413)
0925 10:54:41.400439 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:41.400444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:41.400654 (+ 210us) server_negotiation.cc:407] Connection header received
0925 10:54:41.400725 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:41.400729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:41.400797 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:41.400927 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:41.402170 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:41.402908 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:41.404253 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:41.404684 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:41.404783 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:41.408767 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:41.408774 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:41.408777 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:41.408839 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:41.411539 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:41.412302 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:41.412307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:41.412309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:41.412451 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:41.412812 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:41.412815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:41.412817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:41.413172 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:41.413365 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:41.413703 (+ 338us) server_negotiation.cc:299] Negotiation successful
0925 10:54:41.413897 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260925 10:54:41.414770 3148 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46212
I20260925 10:54:41.415977 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 3209
I20260925 10:54:41.428279 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:41.416971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48554 (local address 127.31.62.129:40413)
0925 10:54:41.417169 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:41.417175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:41.417368 (+ 193us) server_negotiation.cc:407] Connection header received
0925 10:54:41.417513 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:41.417517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:41.417580 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:41.417656 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:41.418819 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:41.419519 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:41.420768 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:41.421209 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:41.421307 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:41.423468 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:41.423473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:41.423476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:41.423510 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:41.426050 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:41.426774 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:41.426778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:41.426780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:41.426920 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:41.427236 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:41.427239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:41.427242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:41.427482 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:41.427644 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:41.427866 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 10:54:41.428026 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 10:54:41.428831 3332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:41.510067 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 3209
I20260925 10:54:41.523052 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 3343
Sep 25 10:54:41 dist-test-slave-nx7s krb5kdc[3103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:54:41.538924 3616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:41.523920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54392 (local address 127.31.62.130:34819)
0925 10:54:41.524434 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:41.524441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:41.524469 (+ 28us) server_negotiation.cc:407] Connection header received
0925 10:54:41.524531 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:41.524538 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:41.524753 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:41.524956 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:41.526105 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:41.527224 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:41.528520 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:41.529240 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:41.529350 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:41.533145 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:41.533158 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:41.533174 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:41.533218 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:41.536092 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:41.536728 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:41.536735 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:41.536738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:41.537123 (+ 385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:41.537535 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:41.537537 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:41.537540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:41.537887 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:41.538102 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:41.538409 (+ 307us) server_negotiation.cc:299] Negotiation successful
0925 10:54:41.538641 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":243,"threads_started":1}
I20260925 10:54:41.540194 3466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:41.621181 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 3343
I20260925 10:54:41.633662 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 3476
Sep 25 10:54:41 dist-test-slave-nx7s krb5kdc[3103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:54:41.649924 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:41.634589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43746 (local address 127.31.62.131:43375)
0925 10:54:41.635176 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:41.635183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:41.635213 (+ 30us) server_negotiation.cc:407] Connection header received
0925 10:54:41.635272 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:41.635278 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:41.635486 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:41.635627 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:41.636891 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:41.637926 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:41.639325 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:41.639906 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:41.640013 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:41.644406 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:41.644421 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:41.644437 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:41.644479 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:41.647284 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:41.647955 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:41.647962 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:41.647964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:41.648086 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:41.648386 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:41.648390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:41.648393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:41.648797 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:41.649059 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:41.649434 (+ 375us) server_negotiation.cc:299] Negotiation successful
0925 10:54:41.649639 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":285,"threads_started":1}
I20260925 10:54:41.651269 3599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:41.733372 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 3476
I20260925 10:54:41.745975 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 3116
I20260925 10:54:41.747217 3178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:41.850615 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 3116
2026-09-25T10:54:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3750 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:54:41 dist-test-slave-nx7s krb5kdc[3623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:54:41 dist-test-slave-nx7s krb5kdc[3623](info): set up 2 sockets
Sep 25 10:54:41 dist-test-slave-nx7s krb5kdc[3623](info): commencing operation
krb5kdc: starting...
W20260925 10:54:42.853538 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.007s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:54:42 dist-test-slave-nx7s krb5kdc[3623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:54:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:54:43.048659 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:34943
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:42895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:34943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:43.371060 3636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:43.371376 3636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:43.371510 3636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:43.378957 3636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:43.379087 3636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:43.379161 3636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:43.379230 3636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:43.379278 3636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:54:43.389525 3636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:42895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:34943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:34943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:43.390857 3636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:43.392181 3636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:43.403582 3644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:43.403678 3641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:43.403673 3642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:43.404803 3636 server_base.cc:1034] running on GCE node
I20260925 10:54:43.405697 3636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:43.407271 3636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:43.408490 3636 hybrid_clock.cc:648] HybridClock initialized: now 1790333683408448 us; error 55 us; skew 500 ppm
I20260925 10:54:43.408879 3636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:43 dist-test-slave-nx7s krb5kdc[3623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333683, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:43.411978 3636 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:54:43.413249 3636 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:43.413452 3636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:43.413908 3636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:43.417128 3636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "d634ceefbcc9496d84d16170d4ab4992"
format_stamp: "Formatted at 2026-09-25 10:54:43 on dist-test-slave-nx7s"
server_key: "6d71ca6875cf34f5de0f653bd64fa00f"
server_key_iv: "c1b5f57ad30cb8770e53e6123b1c14ce"
server_key_version: "encryptionkey@0"
I20260925 10:54:43.417954 3636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "d634ceefbcc9496d84d16170d4ab4992"
format_stamp: "Formatted at 2026-09-25 10:54:43 on dist-test-slave-nx7s"
server_key: "6d71ca6875cf34f5de0f653bd64fa00f"
server_key_iv: "c1b5f57ad30cb8770e53e6123b1c14ce"
server_key_version: "encryptionkey@0"
I20260925 10:54:43.423259 3636 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260925 10:54:43.426978 3650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:43.428414 3636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:54:43.428593 3636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "d634ceefbcc9496d84d16170d4ab4992"
format_stamp: "Formatted at 2026-09-25 10:54:43 on dist-test-slave-nx7s"
server_key: "6d71ca6875cf34f5de0f653bd64fa00f"
server_key_iv: "c1b5f57ad30cb8770e53e6123b1c14ce"
server_key_version: "encryptionkey@0"
I20260925 10:54:43.428814 3636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:43.444101 3636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:43.448745 3636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:43.449054 3636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:43.465997 3636 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:34943
I20260925 10:54:43.465979 3701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:34943 every 8 connection(s)
I20260925 10:54:43.466944 3636 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:43.470675 3702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:43.477238 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 3636
I20260925 10:54:43.477458 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:43.478252 31994 external_mini_cluster.cc:1428] Setting key 475be0425fe51edff4254f11fc658a25
I20260925 10:54:43.482484 3702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992: Bootstrap starting.
I20260925 10:54:43.485715 3702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:43.486862 3702 log.cc:824] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:43.489669 3702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992: No bootstrap required, opened a new log
Sep 25 10:54:43 dist-test-slave-nx7s krb5kdc[3623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:43.497345 3702 raft_consensus.cc:348] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d634ceefbcc9496d84d16170d4ab4992" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34943 } }
I20260925 10:54:43.497746 3702 raft_consensus.cc:374] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:43.497839 3702 raft_consensus.cc:729] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d634ceefbcc9496d84d16170d4ab4992, State: Initialized, Role: FOLLOWER
I20260925 10:54:43.498473 3702 consensus_queue.cc:260] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d634ceefbcc9496d84d16170d4ab4992" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34943 } }
I20260925 10:54:43.498700 3702 raft_consensus.cc:388] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:43.498853 3702 raft_consensus.cc:482] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:43.499011 3702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:43.500890 3705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:43.480668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51116 (local address 127.31.62.190:34943)
0925 10:54:43.481618 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:43.481631 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:43.481677 (+ 46us) server_negotiation.cc:407] Connection header received
0925 10:54:43.483529 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:43.483554 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:43.483988 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:43.484411 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:43.485947 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:43.487404 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:43.488678 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:43.489400 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:43.489576 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:43.493328 (+ 3752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:43.493337 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:43.493358 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:43.493399 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:43.496577 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:43.497385 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:43.497392 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:43.497399 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:43.497572 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:43.498011 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:43.498015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:43.498017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:43.498604 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:43.498882 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:43.499341 (+ 459us) server_negotiation.cc:299] Negotiation successful
0925 10:54:43.499781 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":407,"threads_started":1}
I20260925 10:54:43.501549 3702 raft_consensus.cc:504] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d634ceefbcc9496d84d16170d4ab4992" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34943 } }
I20260925 10:54:43.502061 3702 leader_election.cc:302] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d634ceefbcc9496d84d16170d4ab4992; no voters:
I20260925 10:54:43.502664 3702 leader_election.cc:288] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:43.502810 3707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:43.504594 3707 raft_consensus.cc:686] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [term 1 LEADER]: Becoming Leader. State: Replica: d634ceefbcc9496d84d16170d4ab4992, State: Running, Role: LEADER
I20260925 10:54:43.505103 3707 consensus_queue.cc:237] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d634ceefbcc9496d84d16170d4ab4992" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34943 } }
I20260925 10:54:43.505508 3702 sys_catalog.cc:564] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:43.507943 3709 sys_catalog.cc:455] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d634ceefbcc9496d84d16170d4ab4992. Latest consensus state: current_term: 1 leader_uuid: "d634ceefbcc9496d84d16170d4ab4992" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d634ceefbcc9496d84d16170d4ab4992" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34943 } } }
I20260925 10:54:43.508008 3708 sys_catalog.cc:455] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d634ceefbcc9496d84d16170d4ab4992" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d634ceefbcc9496d84d16170d4ab4992" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34943 } } }
I20260925 10:54:43.508288 3709 sys_catalog.cc:458] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:43.508351 3708 sys_catalog.cc:458] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:43.510843 3715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:43.516072 3715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:43.524840 3715 catalog_manager.cc:1347] Generated new cluster ID: 741cf46a41a043f891e5314b2f2d4ae3
I20260925 10:54:43.524981 3715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:43.538982 3715 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:43.540270 3715 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:43.545903 3715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992: Generated new TSK 0
I20260925 10:54:43.546962 3715 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:54:43.602649 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34943
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:42895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:43.923888 3728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:43.924193 3728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:43.924333 3728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:43.931596 3728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:43.931726 3728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:43.931880 3728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:54:43.941963 3728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:42895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:43.943295 3728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:43.944685 3728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:43.957377 3734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:43.957439 3733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:43.957666 3736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:43.958287 3728 server_base.cc:1034] running on GCE node
I20260925 10:54:43.958918 3728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:43.959903 3728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:43.961145 3728 hybrid_clock.cc:648] HybridClock initialized: now 1790333683961075 us; error 47 us; skew 500 ppm
I20260925 10:54:43.961499 3728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:43 dist-test-slave-nx7s krb5kdc[3623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333683, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:43.964696 3728 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:54:43.965833 3728 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:43.965922 3728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:43.966190 3728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:43.969201 3728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "35aa0b3fd40f4085860149f094afc2c2"
format_stamp: "Formatted at 2026-09-25 10:54:43 on dist-test-slave-nx7s"
server_key: "e148434a6317392249758ae810968c0d"
server_key_iv: "6ac6d2e3349ada568f90d2619f31fa07"
server_key_version: "encryptionkey@0"
I20260925 10:54:43.969935 3728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "35aa0b3fd40f4085860149f094afc2c2"
format_stamp: "Formatted at 2026-09-25 10:54:43 on dist-test-slave-nx7s"
server_key: "e148434a6317392249758ae810968c0d"
server_key_iv: "6ac6d2e3349ada568f90d2619f31fa07"
server_key_version: "encryptionkey@0"
I20260925 10:54:43.975135 3728 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:54:43.979024 3742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:43.980655 3728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:54:43.980834 3728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "35aa0b3fd40f4085860149f094afc2c2"
format_stamp: "Formatted at 2026-09-25 10:54:43 on dist-test-slave-nx7s"
server_key: "e148434a6317392249758ae810968c0d"
server_key_iv: "6ac6d2e3349ada568f90d2619f31fa07"
server_key_version: "encryptionkey@0"
I20260925 10:54:43.981137 3728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:43.999728 3728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:44.003957 3728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:44.004225 3728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:44.005247 3728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:44.006731 3728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:44.006844 3728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:44.006963 3728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:44.007035 3728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:44.037334 3728 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:43527
I20260925 10:54:44.037386 3854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:43527 every 8 connection(s)
I20260925 10:54:44.038266 3728 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:54:44.043223 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 3728
I20260925 10:54:44.043409 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:54:44.044056 31994 external_mini_cluster.cc:1428] Setting key cb626960493d1308635fa0c23abca627
Sep 25 10:54:44 dist-test-slave-nx7s krb5kdc[3623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333683, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:44.059928 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:44.040657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:52513 (local address 127.31.62.190:34943)
0925 10:54:44.041178 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:44.041184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:44.042185 (+ 1001us) server_negotiation.cc:407] Connection header received
0925 10:54:44.044159 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:44.044166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:44.044254 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:44.044381 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:44.046489 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:44.047493 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:44.049342 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:44.049904 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:44.050024 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:44.053941 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:44.053946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:44.053950 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:44.053984 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:44.056445 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:44.057468 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:44.057472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:44.057474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:44.057599 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:44.058105 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:44.058109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:44.058113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:44.058400 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:44.058509 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:44.059482 (+ 973us) server_negotiation.cc:299] Negotiation successful
0925 10:54:44.059650 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":293,"threads_started":1}
I20260925 10:54:44.061635 3858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:44.041306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34943 (local address 127.31.62.129:52513)
0925 10:54:44.042017 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:44.042056 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:44.043823 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:44.044643 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:44.044656 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:44.045567 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:44.046314 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:44.046336 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:44.047673 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:44.047678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:44.049191 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:44.049205 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:44.050169 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:44.050176 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:44.050390 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:44.051151 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:44.051176 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:44.053682 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:44.056599 (+ 2917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:44.056610 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:44.056631 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:44.057336 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:44.057700 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:44.057705 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:44.057709 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:44.057962 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:44.058531 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:44.058541 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:44.059027 (+ 486us) client_negotiation.cc:769] Sending connection context
0925 10:54:44.059387 (+ 360us) client_negotiation.cc:240] Negotiation successful
0925 10:54:44.059992 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":289,"threads_started":1}
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
I20260925 10:54:44.063460 3855 heartbeater.cc:347] Connected to a master server at 127.31.62.190:34943
I20260925 10:54:44.063794 3855 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:44.064488 3855 heartbeater.cc:511] Master 127.31.62.190:34943 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
I20260925 10:54:44.066687 3667 ts_manager.cc:194] Registered new tserver with Master: 35aa0b3fd40f4085860149f094afc2c2 (127.31.62.129:43527)
I20260925 10:54:44.068121 3667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:52513
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:54:44.088881 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34943
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:42895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:44.410547 3861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:44.410859 3861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:44.410984 3861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:44.418442 3861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:44.418574 3861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:44.418725 3861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:54:44.428643 3861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:42895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:44.430053 3861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:44.431418 3861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:44.445195 3869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:44.445254 3866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:44.445233 3867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:44.446038 3861 server_base.cc:1034] running on GCE node
I20260925 10:54:44.446710 3861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:44.447799 3861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:44.449010 3861 hybrid_clock.cc:648] HybridClock initialized: now 1790333684448939 us; error 60 us; skew 500 ppm
I20260925 10:54:44.449461 3861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:44 dist-test-slave-nx7s krb5kdc[3623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333684, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:44.452786 3861 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:54:44.454150 3861 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:44.454250 3861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:44.454559 3861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:44.457677 3861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "eb2ca281d5c24ee8a134ce969122ff73"
format_stamp: "Formatted at 2026-09-25 10:54:44 on dist-test-slave-nx7s"
server_key: "ce273482362e760ea0cab44df1725b20"
server_key_iv: "8395807fe14ce4ddd7e90340f36d12fc"
server_key_version: "encryptionkey@0"
I20260925 10:54:44.458510 3861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "eb2ca281d5c24ee8a134ce969122ff73"
format_stamp: "Formatted at 2026-09-25 10:54:44 on dist-test-slave-nx7s"
server_key: "ce273482362e760ea0cab44df1725b20"
server_key_iv: "8395807fe14ce4ddd7e90340f36d12fc"
server_key_version: "encryptionkey@0"
I20260925 10:54:44.463982 3861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260925 10:54:44.468217 3875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:44.469920 3861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:54:44.470098 3861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "eb2ca281d5c24ee8a134ce969122ff73"
format_stamp: "Formatted at 2026-09-25 10:54:44 on dist-test-slave-nx7s"
server_key: "ce273482362e760ea0cab44df1725b20"
server_key_iv: "8395807fe14ce4ddd7e90340f36d12fc"
server_key_version: "encryptionkey@0"
I20260925 10:54:44.470341 3861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:44.499032 3861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:44.503252 3861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:44.503497 3861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:44.504419 3861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:44.505755 3861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:44.505837 3861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:44.505930 3861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:44.505992 3861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:44.540436 3861 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:42521
I20260925 10:54:44.540491 3987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:42521 every 8 connection(s)
I20260925 10:54:44.541276 3861 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:54:44.547870 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 3861
I20260925 10:54:44.548033 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:54:44.548453 31994 external_mini_cluster.cc:1428] Setting key e40d1ea81c045c248ae09e67db58710a
Sep 25 10:54:44 dist-test-slave-nx7s krb5kdc[3623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333684, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:44.561587 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:44.543769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:33207 (local address 127.31.62.190:34943)
0925 10:54:44.543983 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:44.543988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:44.545176 (+ 1188us) server_negotiation.cc:407] Connection header received
0925 10:54:44.546995 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:44.547000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:44.547069 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:44.547162 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:44.548933 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:44.549638 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:44.551439 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:44.552051 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:44.552177 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:44.555721 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:44.555727 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:44.555732 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:44.555773 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:44.558456 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:44.559339 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:44.559343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:44.559345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:44.559468 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:44.559926 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:44.559929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:44.559931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:44.560186 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:44.560290 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:44.561151 (+ 861us) server_negotiation.cc:299] Negotiation successful
0925 10:54:44.561344 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 10:54:44.563113 3990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:44.544208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34943 (local address 127.31.62.130:33207)
0925 10:54:44.545014 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:44.545050 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:44.546735 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:44.547472 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:44.547482 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:44.548067 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:44.548717 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:44.548739 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:44.549799 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:44.549805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:44.551289 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:44.551301 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:44.552277 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:44.552282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:44.552478 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:44.553165 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:44.553190 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:44.555457 (+ 2267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:44.558620 (+ 3163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:44.558630 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:44.558650 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:44.559230 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:44.559572 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:44.559576 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:44.559580 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:44.559831 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:44.560296 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:44.560304 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:44.560772 (+ 468us) client_negotiation.cc:769] Sending connection context
0925 10:54:44.561085 (+ 313us) client_negotiation.cc:240] Negotiation successful
0925 10:54:44.561629 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":279,"threads_started":1}
I20260925 10:54:44.564834 3988 heartbeater.cc:347] Connected to a master server at 127.31.62.190:34943
I20260925 10:54:44.565182 3988 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:44.565843 3988 heartbeater.cc:511] Master 127.31.62.190:34943 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
I20260925 10:54:44.567287 3667 ts_manager.cc:194] Registered new tserver with Master: eb2ca281d5c24ee8a134ce969122ff73 (127.31.62.130:42521)
I20260925 10:54:44.568090 3667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:33207
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:54:44.592968 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34943
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:42895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:44.909806 3993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:44.910131 3993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:44.910269 3993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:44.917691 3993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:44.917810 3993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:44.917961 3993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:54:44.928242 3993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:42895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:44.929631 3993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:44.930974 3993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:44.944134 3998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:44.944226 3999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:44.944231 4001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:44.944909 3993 server_base.cc:1034] running on GCE node
I20260925 10:54:44.945598 3993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:44.946674 3993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:44.947890 3993 hybrid_clock.cc:648] HybridClock initialized: now 1790333684947832 us; error 52 us; skew 500 ppm
I20260925 10:54:44.948272 3993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:44 dist-test-slave-nx7s krb5kdc[3623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333684, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:44.955677 3993 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:54:44.956939 3993 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:44.957054 3993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:44.957427 3993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:44.960410 3993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "9c521ba9db1f42049c018556c79278b3"
format_stamp: "Formatted at 2026-09-25 10:54:44 on dist-test-slave-nx7s"
server_key: "0001689560d97498d4a999ec204e5a75"
server_key_iv: "aa01cbb7b0bf766d974c27cc3d5f5066"
server_key_version: "encryptionkey@0"
I20260925 10:54:44.961261 3993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "9c521ba9db1f42049c018556c79278b3"
format_stamp: "Formatted at 2026-09-25 10:54:44 on dist-test-slave-nx7s"
server_key: "0001689560d97498d4a999ec204e5a75"
server_key_iv: "aa01cbb7b0bf766d974c27cc3d5f5066"
server_key_version: "encryptionkey@0"
I20260925 10:54:44.966620 3993 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:54:44.970221 4007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:44.971689 3993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:54:44.971879 3993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "9c521ba9db1f42049c018556c79278b3"
format_stamp: "Formatted at 2026-09-25 10:54:44 on dist-test-slave-nx7s"
server_key: "0001689560d97498d4a999ec204e5a75"
server_key_iv: "aa01cbb7b0bf766d974c27cc3d5f5066"
server_key_version: "encryptionkey@0"
I20260925 10:54:44.972093 3993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:44.995502 3993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:45.000054 3993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:45.000340 3993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:45.001405 3993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:45.002911 3993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:45.003003 3993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:45.003084 3993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:45.003144 3993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:45.032811 3993 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:45457
I20260925 10:54:45.032886 4119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:45457 every 8 connection(s)
I20260925 10:54:45.033674 3993 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:54:45.041059 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 3993
I20260925 10:54:45.041219 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:54:45.041641 31994 external_mini_cluster.cc:1428] Setting key 2a2b42bf4af35eb2fe83b3c60a64705f
Sep 25 10:54:45 dist-test-slave-nx7s krb5kdc[3623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333684, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:45.055397 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.036148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:49383 (local address 127.31.62.190:34943)
0925 10:54:45.036324 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:45.036328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:45.037723 (+ 1395us) server_negotiation.cc:407] Connection header received
0925 10:54:45.040324 (+ 2601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:45.040329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:45.040413 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:45.040496 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:45.042535 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.043619 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.045628 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.046153 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.046259 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.049663 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:45.049668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:45.049671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:45.049705 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:45.052260 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:45.053064 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:45.053069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:45.053072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:45.053214 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:45.053568 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:45.053572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:45.053574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:45.053873 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:45.053987 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:45.054764 (+ 777us) server_negotiation.cc:299] Negotiation successful
0925 10:54:45.054968 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 10:54:45.056224 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.036648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34943 (local address 127.31.62.131:49383)
0925 10:54:45.037530 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:45.037580 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:45.039952 (+ 2372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:45.040776 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:45.040798 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:45.041529 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:45.042346 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:45.042374 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.043785 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.043791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:45.045480 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:45.045494 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.046382 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.046388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:45.046585 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.047219 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:45.047242 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:45.049391 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:45.052407 (+ 3016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:45.052414 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:45.052430 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:45.052936 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:45.053305 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:45.053308 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:45.053310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:45.053484 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:45.053969 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:45.053975 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:45.054348 (+ 373us) client_negotiation.cc:769] Sending connection context
0925 10:54:45.054593 (+ 245us) client_negotiation.cc:240] Negotiation successful
0925 10:54:45.055011 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":410,"threads_started":1}
I20260925 10:54:45.057936 4120 heartbeater.cc:347] Connected to a master server at 127.31.62.190:34943
I20260925 10:54:45.058277 4120 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:45.058897 4120 heartbeater.cc:511] Master 127.31.62.190:34943 requested a full tablet report, sending...
I20260925 10:54:45.060287 3667 ts_manager.cc:194] Registered new tserver with Master: 9c521ba9db1f42049c018556c79278b3 (127.31.62.131:45457)
I20260925 10:54:45.061163 3667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:49383
I20260925 10:54:45.070657 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:54:45.071188 3855 heartbeater.cc:503] Master 127.31.62.190:34943 was elected leader, sending a full tablet report...
I20260925 10:54:45.087764 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.075202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51132 (local address 127.31.62.190:34943)
0925 10:54:45.075405 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:45.075409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:45.075929 (+ 520us) server_negotiation.cc:407] Connection header received
0925 10:54:45.076038 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:45.076043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:45.076124 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:45.076262 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:45.077751 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.078459 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.079728 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.080164 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.080265 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.082606 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:45.082611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:45.082614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:45.082646 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:45.085284 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:45.085998 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:45.086002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:45.086004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:45.086135 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:45.086457 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:45.086460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:45.086462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:45.086822 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:45.086996 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:45.087373 (+ 377us) server_negotiation.cc:299] Negotiation successful
0925 10:54:45.087535 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 10:54:45.091897 3667 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:54:45.098655 3667 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:54:45.127830 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.111043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:45457 (local address 127.0.0.1:59370)
0925 10:54:45.115217 (+ 4174us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:45.115236 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:45.115377 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:45.116516 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:45.116525 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:45.116559 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:45.117031 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:45.117044 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.120570 (+ 3526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.120576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:45.124612 (+ 4036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:45.124632 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.126665 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.126683 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:45.126936 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.126997 (+ 61us) client_negotiation.cc:769] Sending connection context
0925 10:54:45.127204 (+ 207us) client_negotiation.cc:240] Negotiation successful
0925 10:54:45.127429 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4044,"thread_start_us":2589,"threads_started":1}
I20260925 10:54:45.128721 4135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.111366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59370 (local address 127.31.62.131:45457)
0925 10:54:45.113447 (+ 2081us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:45.113459 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:45.115513 (+ 2054us) server_negotiation.cc:407] Connection header received
0925 10:54:45.115632 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:45.115641 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:45.115959 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:45.116163 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:45.117301 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.120379 (+ 3078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.124878 (+ 4499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.126309 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.126435 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.126688 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:45.128215 (+ 1527us) server_negotiation.cc:299] Negotiation successful
0925 10:54:45.128388 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1871,"thread_start_us":1689,"threads_started":1}
I20260925 10:54:45.133272 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.110766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48556 (local address 127.31.62.129:43527)
0925 10:54:45.125999 (+ 15233us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:45.126007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:45.126037 (+ 30us) server_negotiation.cc:407] Connection header received
0925 10:54:45.126123 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:45.126129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:45.126381 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:45.127307 (+ 926us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:45.127529 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.128988 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.131266 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.132188 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.132318 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.132545 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:45.132732 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 10:54:45.132930 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15069,"thread_start_us":14861,"threads_started":1}
I20260925 10:54:45.133302 4132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.110650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:43527 (local address 127.0.0.1:48556)
0925 10:54:45.114851 (+ 4201us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:45.114901 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:45.115073 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:45.126599 (+ 11526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:45.126603 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:45.126638 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:45.127044 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:45.127055 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.129128 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.129134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:45.131135 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:45.131144 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.132385 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.132391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:45.132554 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.132587 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 10:54:45.132676 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 10:54:45.132819 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3955,"mutex_wait_us":55,"thread_start_us":3815,"threads_started":1}
I20260925 10:54:45.133579 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.115367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:42521 (local address 127.0.0.1:42808)
0925 10:54:45.124614 (+ 9247us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:45.124637 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:45.124775 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:45.126333 (+ 1558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:45.126338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:45.126365 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:45.126775 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:45.126796 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.128612 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.128617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:45.130805 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:45.130820 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.132793 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.132799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:45.132995 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.133026 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 10:54:45.133111 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 10:54:45.133238 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9064,"mutex_wait_us":35,"thread_start_us":9527,"threads_started":1}
I20260925 10:54:45.132954 4055 tablet_service.cc:1467] Processing CreateTablet for tablet 5d9e205896f14b7380927a91cdb3cc36 (DEFAULT_TABLE table=test-table [id=1c1f367ac9ea48298ef40ce7ce048dd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:45.134280 4136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.115063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42808 (local address 127.31.62.130:42521)
0925 10:54:45.125472 (+ 10409us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:45.125482 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:45.125515 (+ 33us) server_negotiation.cc:407] Connection header received
0925 10:54:45.125797 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:45.125808 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:45.126106 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:45.126325 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:45.127166 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.128489 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.130974 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.132325 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.132457 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.132639 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:45.133805 (+ 1166us) server_negotiation.cc:299] Negotiation successful
0925 10:54:45.133982 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10261,"thread_start_us":10123,"threads_started":1}
I20260925 10:54:45.134572 4055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d9e205896f14b7380927a91cdb3cc36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:45.137679 3923 tablet_service.cc:1467] Processing CreateTablet for tablet 5d9e205896f14b7380927a91cdb3cc36 (DEFAULT_TABLE table=test-table [id=1c1f367ac9ea48298ef40ce7ce048dd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:45.137778 3790 tablet_service.cc:1467] Processing CreateTablet for tablet 5d9e205896f14b7380927a91cdb3cc36 (DEFAULT_TABLE table=test-table [id=1c1f367ac9ea48298ef40ce7ce048dd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:45.139093 3923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d9e205896f14b7380927a91cdb3cc36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:45.139107 3790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d9e205896f14b7380927a91cdb3cc36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:45.151309 4138 tablet_bootstrap.cc:492] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73: Bootstrap starting.
I20260925 10:54:45.152947 4140 tablet_bootstrap.cc:492] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2: Bootstrap starting.
I20260925 10:54:45.154536 4139 tablet_bootstrap.cc:492] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3: Bootstrap starting.
I20260925 10:54:45.154809 4138 tablet_bootstrap.cc:654] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:45.156157 4138 log.cc:824] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:45.156809 4140 tablet_bootstrap.cc:654] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:45.157692 4139 tablet_bootstrap.cc:654] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:45.157967 4140 log.cc:824] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:45.158691 4139 log.cc:824] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:45.159332 4138 tablet_bootstrap.cc:492] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73: No bootstrap required, opened a new log
I20260925 10:54:45.159626 4138 ts_tablet_manager.cc:1397] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260925 10:54:45.161150 4140 tablet_bootstrap.cc:492] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2: No bootstrap required, opened a new log
I20260925 10:54:45.161460 4140 ts_tablet_manager.cc:1397] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260925 10:54:45.167488 4139 tablet_bootstrap.cc:492] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3: No bootstrap required, opened a new log
I20260925 10:54:45.167791 4139 ts_tablet_manager.cc:1397] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260925 10:54:45.170608 4138 raft_consensus.cc:348] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35aa0b3fd40f4085860149f094afc2c2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 43527 } } peers { permanent_uuid: "eb2ca281d5c24ee8a134ce969122ff73" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 42521 } } peers { permanent_uuid: "9c521ba9db1f42049c018556c79278b3" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 45457 } }
I20260925 10:54:45.171109 4138 raft_consensus.cc:374] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:45.171247 4138 raft_consensus.cc:729] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb2ca281d5c24ee8a134ce969122ff73, State: Initialized, Role: FOLLOWER
I20260925 10:54:45.171787 4140 raft_consensus.cc:348] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35aa0b3fd40f4085860149f094afc2c2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 43527 } } peers { permanent_uuid: "eb2ca281d5c24ee8a134ce969122ff73" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 42521 } } peers { permanent_uuid: "9c521ba9db1f42049c018556c79278b3" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 45457 } }
I20260925 10:54:45.172086 4138 consensus_queue.cc:260] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35aa0b3fd40f4085860149f094afc2c2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 43527 } } peers { permanent_uuid: "eb2ca281d5c24ee8a134ce969122ff73" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 42521 } } peers { permanent_uuid: "9c521ba9db1f42049c018556c79278b3" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 45457 } }
I20260925 10:54:45.172421 4140 raft_consensus.cc:374] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:45.172544 4140 raft_consensus.cc:729] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35aa0b3fd40f4085860149f094afc2c2, State: Initialized, Role: FOLLOWER
I20260925 10:54:45.173383 4140 consensus_queue.cc:260] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35aa0b3fd40f4085860149f094afc2c2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 43527 } } peers { permanent_uuid: "eb2ca281d5c24ee8a134ce969122ff73" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 42521 } } peers { permanent_uuid: "9c521ba9db1f42049c018556c79278b3" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 45457 } }
I20260925 10:54:45.177031 4139 raft_consensus.cc:348] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35aa0b3fd40f4085860149f094afc2c2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 43527 } } peers { permanent_uuid: "eb2ca281d5c24ee8a134ce969122ff73" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 42521 } } peers { permanent_uuid: "9c521ba9db1f42049c018556c79278b3" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 45457 } }
I20260925 10:54:45.177618 4139 raft_consensus.cc:374] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:45.177757 4139 raft_consensus.cc:729] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c521ba9db1f42049c018556c79278b3, State: Initialized, Role: FOLLOWER
I20260925 10:54:45.177886 3988 heartbeater.cc:503] Master 127.31.62.190:34943 was elected leader, sending a full tablet report...
I20260925 10:54:45.178562 4139 consensus_queue.cc:260] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35aa0b3fd40f4085860149f094afc2c2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 43527 } } peers { permanent_uuid: "eb2ca281d5c24ee8a134ce969122ff73" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 42521 } } peers { permanent_uuid: "9c521ba9db1f42049c018556c79278b3" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 45457 } }
I20260925 10:54:45.179057 4138 ts_tablet_manager.cc:1428] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73: Time spent starting tablet: real 0.019s user 0.016s sys 0.004s
I20260925 10:54:45.179761 4120 heartbeater.cc:503] Master 127.31.62.190:34943 was elected leader, sending a full tablet report...
I20260925 10:54:45.180404 4139 ts_tablet_manager.cc:1428] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260925 10:54:45.181773 4140 ts_tablet_manager.cc:1428] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2: Time spent starting tablet: real 0.020s user 0.012s sys 0.004s
W20260925 10:54:45.285140 4121 tablet.cc:2369] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:54:45.287258 4144 raft_consensus.cc:482] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:54:45.287582 4144 raft_consensus.cc:504] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35aa0b3fd40f4085860149f094afc2c2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 43527 } } peers { permanent_uuid: "eb2ca281d5c24ee8a134ce969122ff73" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 42521 } } peers { permanent_uuid: "9c521ba9db1f42049c018556c79278b3" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 45457 } }
I20260925 10:54:45.289266 4144 leader_election.cc:288] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35aa0b3fd40f4085860149f094afc2c2 (127.31.62.129:43527), 9c521ba9db1f42049c018556c79278b3 (127.31.62.131:45457)
W20260925 10:54:45.290413 3856 tablet.cc:2369] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:54:45.293110 3989 tablet.cc:2369] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:54:45.296301 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.289686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:39417 (local address 127.31.62.129:43527)
0925 10:54:45.289881 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:45.289887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:45.290793 (+ 906us) server_negotiation.cc:407] Connection header received
0925 10:54:45.291018 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:45.291022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:45.291078 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:45.291222 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:45.291799 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.292641 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.294604 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.295363 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.295485 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.295558 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:45.295939 (+ 381us) server_negotiation.cc:299] Negotiation successful
0925 10:54:45.296062 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 10:54:45.296320 4135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.289689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:49305 (local address 127.31.62.131:45457)
0925 10:54:45.289881 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:45.289887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:45.290520 (+ 633us) server_negotiation.cc:407] Connection header received
0925 10:54:45.290827 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:45.290833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:45.290905 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:45.290964 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:54:45.291669 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.292396 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.294651 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.295428 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.295583 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.295653 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:45.295933 (+ 280us) server_negotiation.cc:299] Negotiation successful
0925 10:54:45.296060 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 10:54:45.296460 4147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.289605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:45457 (local address 127.31.62.130:49305)
0925 10:54:45.290380 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:45.290410 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:45.290641 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:45.291171 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:45.291175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:45.291199 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:45.291522 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:45.291531 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.292520 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.292525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:45.294499 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:45.294511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.295634 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.295638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:45.295771 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.295814 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 10:54:45.295896 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 10:54:45.296043 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":37,"thread_start_us":386,"threads_started":1}
I20260925 10:54:45.296499 4148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.289830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:43527 (local address 127.31.62.130:39417)
0925 10:54:45.290695 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:45.290712 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:45.290882 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:45.291253 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:45.291257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:45.291295 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:54:45.291657 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:45.291667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.292773 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.292777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:45.294448 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:45.294460 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.295586 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.295592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:45.295771 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.295812 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 10:54:45.295888 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 10:54:45.296049 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":32,"thread_start_us":561,"threads_started":1}
I20260925 10:54:45.297420 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d9e205896f14b7380927a91cdb3cc36" candidate_uuid: "eb2ca281d5c24ee8a134ce969122ff73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c521ba9db1f42049c018556c79278b3" is_pre_election: true
I20260925 10:54:45.297498 3810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d9e205896f14b7380927a91cdb3cc36" candidate_uuid: "eb2ca281d5c24ee8a134ce969122ff73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35aa0b3fd40f4085860149f094afc2c2" is_pre_election: true
I20260925 10:54:45.297899 4075 raft_consensus.cc:2454] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb2ca281d5c24ee8a134ce969122ff73 in term 0.
I20260925 10:54:45.297976 3810 raft_consensus.cc:2454] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb2ca281d5c24ee8a134ce969122ff73 in term 0.
I20260925 10:54:45.298517 3879 leader_election.cc:302] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c521ba9db1f42049c018556c79278b3, eb2ca281d5c24ee8a134ce969122ff73; no voters:
I20260925 10:54:45.298882 4144 raft_consensus.cc:2781] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:54:45.299021 4144 raft_consensus.cc:482] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:54:45.299223 4144 raft_consensus.cc:3037] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:45.301397 4144 raft_consensus.cc:504] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35aa0b3fd40f4085860149f094afc2c2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 43527 } } peers { permanent_uuid: "eb2ca281d5c24ee8a134ce969122ff73" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 42521 } } peers { permanent_uuid: "9c521ba9db1f42049c018556c79278b3" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 45457 } }
I20260925 10:54:45.301991 4144 leader_election.cc:288] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [CANDIDATE]: Term 1 election: Requested vote from peers 35aa0b3fd40f4085860149f094afc2c2 (127.31.62.129:43527), 9c521ba9db1f42049c018556c79278b3 (127.31.62.131:45457)
I20260925 10:54:45.302397 3810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d9e205896f14b7380927a91cdb3cc36" candidate_uuid: "eb2ca281d5c24ee8a134ce969122ff73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35aa0b3fd40f4085860149f094afc2c2"
I20260925 10:54:45.302462 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d9e205896f14b7380927a91cdb3cc36" candidate_uuid: "eb2ca281d5c24ee8a134ce969122ff73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c521ba9db1f42049c018556c79278b3"
I20260925 10:54:45.302574 3810 raft_consensus.cc:3037] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:45.302655 4075 raft_consensus.cc:3037] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:45.305097 3810 raft_consensus.cc:2454] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb2ca281d5c24ee8a134ce969122ff73 in term 1.
I20260925 10:54:45.305128 4075 raft_consensus.cc:2454] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb2ca281d5c24ee8a134ce969122ff73 in term 1.
I20260925 10:54:45.305512 3878 leader_election.cc:302] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35aa0b3fd40f4085860149f094afc2c2, eb2ca281d5c24ee8a134ce969122ff73; no voters:
I20260925 10:54:45.305847 4144 raft_consensus.cc:2781] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:45.306346 4144 raft_consensus.cc:686] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [term 1 LEADER]: Becoming Leader. State: Replica: eb2ca281d5c24ee8a134ce969122ff73, State: Running, Role: LEADER
I20260925 10:54:45.306849 4144 consensus_queue.cc:237] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35aa0b3fd40f4085860149f094afc2c2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 43527 } } peers { permanent_uuid: "eb2ca281d5c24ee8a134ce969122ff73" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 42521 } } peers { permanent_uuid: "9c521ba9db1f42049c018556c79278b3" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 45457 } }
I20260925 10:54:45.310825 3667 catalog_manager.cc:5495] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb2ca281d5c24ee8a134ce969122ff73 (127.31.62.130). New cstate: current_term: 1 leader_uuid: "eb2ca281d5c24ee8a134ce969122ff73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35aa0b3fd40f4085860149f094afc2c2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb2ca281d5c24ee8a134ce969122ff73" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 42521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c521ba9db1f42049c018556c79278b3" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 45457 } health_report { overall_health: UNKNOWN } } }
I20260925 10:54:45.363664 4136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.357809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42812 (local address 127.31.62.130:42521)
0925 10:54:45.358095 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:45.358101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:45.358121 (+ 20us) server_negotiation.cc:407] Connection header received
0925 10:54:45.358263 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:45.358268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:45.358377 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:45.358541 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:54:45.359011 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.360036 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.361571 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.361970 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.362068 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.362602 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:54:45.363115 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:54:45.363229 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:45.363381 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 10:54:45.363452 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260925 10:54:45.372418 4075 raft_consensus.cc:1261] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3 [term 1 FOLLOWER]: Refusing update from remote peer eb2ca281d5c24ee8a134ce969122ff73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:45.372416 3810 raft_consensus.cc:1261] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2 [term 1 FOLLOWER]: Refusing update from remote peer eb2ca281d5c24ee8a134ce969122ff73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:45.373317 4144 consensus_queue.cc:1035] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35aa0b3fd40f4085860149f094afc2c2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:45.373898 4144 consensus_queue.cc:1035] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c521ba9db1f42049c018556c79278b3" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 45457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:45.402103 3666 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:45.402410 3666 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:45.405902 3666 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d634ceefbcc9496d84d16170d4ab4992: Sending DeleteTablet for 3 replicas of tablet 5d9e205896f14b7380927a91cdb3cc36
I20260925 10:54:45.407166 3923 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d9e205896f14b7380927a91cdb3cc36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:45 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:42808
I20260925 10:54:45.407542 3790 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d9e205896f14b7380927a91cdb3cc36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:45 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:48556
I20260925 10:54:45.407899 4055 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d9e205896f14b7380927a91cdb3cc36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:45 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:59370
I20260925 10:54:45.408541 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 3728
Sep 25 10:54:45 dist-test-slave-nx7s krb5kdc[3623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:54:45.430727 4163 tablet_replica.cc:330] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2: stopping tablet replica
I20260925 10:54:45.431344 4163 raft_consensus.cc:2229] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:54:45.431594 4164 tablet_replica.cc:330] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3: stopping tablet replica
I20260925 10:54:45.432004 4164 raft_consensus.cc:2229] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:54:45.432168 4163 raft_consensus.cc:2258] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:45.432492 4164 raft_consensus.cc:2258] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:45.434885 4163 ts_tablet_manager.cc:1905] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:45.434880 4164 ts_tablet_manager.cc:1905] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:45.437434 4162 tablet_replica.cc:330] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73: stopping tablet replica
I20260925 10:54:45.437875 4162 raft_consensus.cc:2229] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:54:45.438553 4162 raft_consensus.cc:2258] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:45.441038 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.409980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48568 (local address 127.31.62.129:43527)
0925 10:54:45.410122 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:45.410128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:45.419578 (+ 9450us) server_negotiation.cc:407] Connection header received
0925 10:54:45.420077 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:45.420084 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:45.420172 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:45.420246 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:45.422035 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.423245 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.425209 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.425923 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.426020 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.431617 (+ 5597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:45.431638 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:45.431657 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:45.431701 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:45.435459 (+ 3758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:45.436378 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:45.436387 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:45.436391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:45.436583 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:45.438941 (+ 2358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:45.438947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:45.438950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:45.439424 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:45.439753 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:45.440371 (+ 618us) server_negotiation.cc:299] Negotiation successful
0925 10:54:45.440637 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 10:54:45.442405 4164 ts_tablet_manager.cc:1918] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:54:45.442492 3850 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:45.442677 4164 log.cc:1196] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/5d9e205896f14b7380927a91cdb3cc36
I20260925 10:54:45.443346 4164 ts_tablet_manager.cc:1939] T 5d9e205896f14b7380927a91cdb3cc36 P 9c521ba9db1f42049c018556c79278b3: Deleting consensus metadata
I20260925 10:54:45.443608 4162 ts_tablet_manager.cc:1905] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:45.445528 3654 catalog_manager.cc:4841] TS 9c521ba9db1f42049c018556c79278b3 (127.31.62.131:45457): tablet 5d9e205896f14b7380927a91cdb3cc36 (table test-table [id=1c1f367ac9ea48298ef40ce7ce048dd3]) successfully deleted
I20260925 10:54:45.451603 4162 ts_tablet_manager.cc:1918] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:54:45.451809 4162 log.cc:1196] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/5d9e205896f14b7380927a91cdb3cc36
I20260925 10:54:45.452428 4162 ts_tablet_manager.cc:1939] T 5d9e205896f14b7380927a91cdb3cc36 P eb2ca281d5c24ee8a134ce969122ff73: Deleting consensus metadata
I20260925 10:54:45.454620 3653 catalog_manager.cc:4841] TS eb2ca281d5c24ee8a134ce969122ff73 (127.31.62.130:42521): tablet 5d9e205896f14b7380927a91cdb3cc36 (table test-table [id=1c1f367ac9ea48298ef40ce7ce048dd3]) successfully deleted
I20260925 10:54:45.547644 4163 ts_tablet_manager.cc:1918] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:54:45.547888 4163 log.cc:1196] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/5d9e205896f14b7380927a91cdb3cc36
I20260925 10:54:45.548326 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 3728
I20260925 10:54:45.548453 4163 ts_tablet_manager.cc:1939] T 5d9e205896f14b7380927a91cdb3cc36 P 35aa0b3fd40f4085860149f094afc2c2: Deleting consensus metadata
W20260925 10:54:45.562729 3653 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.62.129:43527 (error 108)
I20260925 10:54:45.564194 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 3861
I20260925 10:54:45.564762 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.564155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:43527 (local address 127.0.0.1:48584)
0925 10:54:45.564438 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:45.564584 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:43527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
W20260925 10:54:45.565371 3653 catalog_manager.cc:4584] TS 35aa0b3fd40f4085860149f094afc2c2 (127.31.62.129:43527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5d9e205896f14b7380927a91cdb3cc36: Network error: Client connection negotiation failed: client connection to 127.31.62.129:43527: connect: Connection refused (error 111)
Sep 25 10:54:45 dist-test-slave-nx7s krb5kdc[3623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:54:45.579454 4136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.565147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42814 (local address 127.31.62.130:42521)
0925 10:54:45.565303 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:45.565309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:45.565329 (+ 20us) server_negotiation.cc:407] Connection header received
0925 10:54:45.565392 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:45.565401 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:45.565480 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:45.565584 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:45.566936 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.567754 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.568998 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.569445 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.569552 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.573556 (+ 4004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:45.573572 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:45.573587 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:45.573634 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:45.576671 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:45.577387 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:45.577394 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:45.577397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:45.577542 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:45.577932 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:45.577936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:45.577938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:45.578302 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:45.578623 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:45.578897 (+ 274us) server_negotiation.cc:299] Negotiation successful
0925 10:54:45.579099 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 10:54:45.580052 3983 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:45.619017 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.618484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:43527 (local address 127.0.0.1:40530)
0925 10:54:45.618715 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:45.618838 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:43527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260925 10:54:45.620142 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.619717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:43527 (local address 127.0.0.1:40538)
0925 10:54:45.619871 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:45.619969 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:43527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 10:54:45.680984 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.680358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:43527 (local address 127.0.0.1:40540)
0925 10:54:45.680593 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:45.680727 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:43527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260925 10:54:45.682042 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.681704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:43527 (local address 127.0.0.1:40548)
0925 10:54:45.681842 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:45.681917 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:43527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260925 10:54:45.693136 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 3861
I20260925 10:54:45.707828 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 3993
Sep 25 10:54:45 dist-test-slave-nx7s krb5kdc[3623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:54:45.722463 4135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.708531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52716 (local address 127.31.62.131:45457)
0925 10:54:45.708697 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:45.708702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:45.708750 (+ 48us) server_negotiation.cc:407] Connection header received
0925 10:54:45.708986 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:45.708991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:45.709069 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:45.709149 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:45.710387 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.711125 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.712373 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:45.712754 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:45.712895 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:45.717014 (+ 4119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:45.717030 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:45.717047 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:45.717084 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:45.719692 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:45.720386 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:45.720394 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:45.720396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:45.720537 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:45.720966 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:45.720972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:45.720974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:45.721322 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:45.721596 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:45.721922 (+ 326us) server_negotiation.cc:299] Negotiation successful
0925 10:54:45.722127 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 10:54:45.723090 4115 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:45.762966 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.762448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:43527 (local address 127.0.0.1:40550)
0925 10:54:45.762660 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:45.762810 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:43527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260925 10:54:45.764192 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:45.763769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:43527 (local address 127.0.0.1:40560)
0925 10:54:45.763981 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:45.764058 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:43527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260925 10:54:45.810635 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 3993
I20260925 10:54:45.823338 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 3636
I20260925 10:54:45.824528 3697 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:45.929709 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 3636
2026-09-25T10:54:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4079 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-25T10:54:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:45Z Disabled control of system clock
I20260925 10:54:45.988291 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:33657
--webserver_interface=127.31.62.190
--webserver_port=0
--builtin_ntp_servers=127.31.62.148:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:33657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 10:54:46.307404 4174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:46.307700 4174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:46.307787 4174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:46.314968 4174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:46.315080 4174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:46.315125 4174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:46.315181 4174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:46.315223 4174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:54:46.324411 4174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:33657
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:33657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:46.325730 4174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:46.327438 4174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:46.339241 4179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:46.339197 4180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:46.339798 4182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:46.340061 4174 server_base.cc:1034] running on GCE node
I20260925 10:54:46.340941 4174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:46.342450 4174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:46.343688 4174 hybrid_clock.cc:648] HybridClock initialized: now 1790333686343644 us; error 34 us; skew 500 ppm
I20260925 10:54:46.344094 4174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:54:46.346664 4174 webserver.cc:460] Webserver started at http://127.31.62.190:38557/ using document root <none> and password file <none>
I20260925 10:54:46.347424 4174 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:46.347514 4174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:46.347777 4174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:46.350951 4174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "b06530e55a7a4dd8bd171401ef899894"
format_stamp: "Formatted at 2026-09-25 10:54:46 on dist-test-slave-nx7s"
server_key: "30464786de1fd44327e5e24948ca60c7"
server_key_iv: "aeba09816cd6ee7b3ee34ab8313dc7b3"
server_key_version: "encryptionkey@0"
I20260925 10:54:46.351809 4174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "b06530e55a7a4dd8bd171401ef899894"
format_stamp: "Formatted at 2026-09-25 10:54:46 on dist-test-slave-nx7s"
server_key: "30464786de1fd44327e5e24948ca60c7"
server_key_iv: "aeba09816cd6ee7b3ee34ab8313dc7b3"
server_key_version: "encryptionkey@0"
I20260925 10:54:46.357399 4174 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 10:54:46.361133 4188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:46.362603 4174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:54:46.362769 4174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "b06530e55a7a4dd8bd171401ef899894"
format_stamp: "Formatted at 2026-09-25 10:54:46 on dist-test-slave-nx7s"
server_key: "30464786de1fd44327e5e24948ca60c7"
server_key_iv: "aeba09816cd6ee7b3ee34ab8313dc7b3"
server_key_version: "encryptionkey@0"
I20260925 10:54:46.362988 4174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:46.368978 4174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:46.369774 4174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:46.370056 4174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:46.385862 4174 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:33657
I20260925 10:54:46.385879 4239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:33657 every 8 connection(s)
I20260925 10:54:46.387161 4174 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:46.391047 4240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:46.395956 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 4174
I20260925 10:54:46.396169 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:46.396941 31994 external_mini_cluster.cc:1428] Setting key 1a6c6dacf435fe690dcfc86362e04aed
I20260925 10:54:46.402592 4240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894: Bootstrap starting.
I20260925 10:54:46.405280 4243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:46.398930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49584 (local address 127.31.62.190:33657)
0925 10:54:46.399969 (+ 1039us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:46.399983 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:46.400028 (+ 45us) server_negotiation.cc:407] Connection header received
0925 10:54:46.401721 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:46.401745 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:46.402168 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:46.402529 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:46.403517 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:46.403524 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:46.403537 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:46.403559 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:46.403616 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:46.403632 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:46.403859 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:46.404013 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 10:54:46.404277 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":366,"threads_started":1}
I20260925 10:54:46.406052 4240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:46.407204 4240 log.cc:824] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:46.410055 4240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894: No bootstrap required, opened a new log
I20260925 10:54:46.417163 4240 raft_consensus.cc:348] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06530e55a7a4dd8bd171401ef899894" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 33657 } }
I20260925 10:54:46.417538 4240 raft_consensus.cc:374] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:46.417631 4240 raft_consensus.cc:729] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b06530e55a7a4dd8bd171401ef899894, State: Initialized, Role: FOLLOWER
I20260925 10:54:46.418224 4240 consensus_queue.cc:260] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06530e55a7a4dd8bd171401ef899894" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 33657 } }
I20260925 10:54:46.418431 4240 raft_consensus.cc:388] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:46.418552 4240 raft_consensus.cc:482] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:46.418699 4240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:46.421046 4240 raft_consensus.cc:504] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06530e55a7a4dd8bd171401ef899894" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 33657 } }
I20260925 10:54:46.421567 4240 leader_election.cc:302] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b06530e55a7a4dd8bd171401ef899894; no voters:
I20260925 10:54:46.422122 4240 leader_election.cc:288] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:46.422252 4245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:46.422916 4245 raft_consensus.cc:686] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [term 1 LEADER]: Becoming Leader. State: Replica: b06530e55a7a4dd8bd171401ef899894, State: Running, Role: LEADER
I20260925 10:54:46.423508 4245 consensus_queue.cc:237] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06530e55a7a4dd8bd171401ef899894" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 33657 } }
I20260925 10:54:46.424065 4240 sys_catalog.cc:564] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:46.425946 4247 sys_catalog.cc:455] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b06530e55a7a4dd8bd171401ef899894. Latest consensus state: current_term: 1 leader_uuid: "b06530e55a7a4dd8bd171401ef899894" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06530e55a7a4dd8bd171401ef899894" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 33657 } } }
I20260925 10:54:46.426240 4247 sys_catalog.cc:458] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:46.427021 4250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:46.426891 4246 sys_catalog.cc:455] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b06530e55a7a4dd8bd171401ef899894" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06530e55a7a4dd8bd171401ef899894" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 33657 } } }
I20260925 10:54:46.428007 4246 sys_catalog.cc:458] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:46.433128 4250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:46.443874 4250 catalog_manager.cc:1347] Generated new cluster ID: 0375ae8342bb48b98aaa87507bd4e438
I20260925 10:54:46.444160 4250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:46.472659 4250 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:46.473179 4250 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:46.487957 4250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894: Generated new TSK 0
I20260925 10:54:46.489151 4250 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:54:46.507107 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:33657
--builtin_ntp_servers=127.31.62.148:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 10:54:46.836652 4264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:46.837010 4264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:46.837149 4264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:46.845031 4264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:46.845152 4264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:46.845289 4264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:54:46.854694 4264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:46.855985 4264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:46.857793 4264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:46.871611 4270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:46.871645 4272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:46.871680 4269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:46.872495 4264 server_base.cc:1034] running on GCE node
I20260925 10:54:46.873206 4264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:46.874245 4264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:46.875469 4264 hybrid_clock.cc:648] HybridClock initialized: now 1790333686875413 us; error 51 us; skew 500 ppm
I20260925 10:54:46.875820 4264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:54:46.878381 4264 webserver.cc:460] Webserver started at http://127.31.62.129:34695/ using document root <none> and password file <none>
I20260925 10:54:46.879120 4264 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:46.879218 4264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:46.879487 4264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:46.882450 4264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "9160ecec81784d5496606910a48a90b5"
format_stamp: "Formatted at 2026-09-25 10:54:46 on dist-test-slave-nx7s"
server_key: "8d6688425399f43afef0ca7166ed638d"
server_key_iv: "64dabd492e00fb783c790dd94f97566c"
server_key_version: "encryptionkey@0"
I20260925 10:54:46.883245 4264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "9160ecec81784d5496606910a48a90b5"
format_stamp: "Formatted at 2026-09-25 10:54:46 on dist-test-slave-nx7s"
server_key: "8d6688425399f43afef0ca7166ed638d"
server_key_iv: "64dabd492e00fb783c790dd94f97566c"
server_key_version: "encryptionkey@0"
I20260925 10:54:46.888805 4264 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 10:54:46.892602 4278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:46.894321 4264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:54:46.894480 4264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "9160ecec81784d5496606910a48a90b5"
format_stamp: "Formatted at 2026-09-25 10:54:46 on dist-test-slave-nx7s"
server_key: "8d6688425399f43afef0ca7166ed638d"
server_key_iv: "64dabd492e00fb783c790dd94f97566c"
server_key_version: "encryptionkey@0"
I20260925 10:54:46.894680 4264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:46.900487 4264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:46.901396 4264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:46.901631 4264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:46.902582 4264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:46.904040 4264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:46.904160 4264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:46.904273 4264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:46.904348 4264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:46.934654 4264 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:37829
I20260925 10:54:46.934702 4390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:37829 every 8 connection(s)
I20260925 10:54:46.935997 4264 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:54:46.944664 4393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:46.938737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:54967 (local address 127.31.62.190:33657)
0925 10:54:46.939193 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:46.939199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:46.940597 (+ 1398us) server_negotiation.cc:407] Connection header received
0925 10:54:46.942338 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:46.942343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:46.942406 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:46.942509 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:46.943894 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:46.943899 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:46.943902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:46.943922 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:46.943965 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:46.943978 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:46.944086 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:46.944304 (+ 218us) server_negotiation.cc:299] Negotiation successful
0925 10:54:46.944367 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":206,"threads_started":1}
I20260925 10:54:46.945595 4394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:46.939428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:33657 (local address 127.31.62.129:54967)
0925 10:54:46.940428 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:46.940467 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:46.942031 (+ 1564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:46.942721 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:46.942730 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:46.942834 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:46.943491 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:54:46.943494 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:46.943539 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:54:46.943633 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:46.944064 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:46.944071 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:46.944077 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 10:54:46.944361 (+ 284us) client_negotiation.cc:240] Negotiation successful
0925 10:54:46.944592 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":278,"threads_started":1}
I20260925 10:54:46.946417 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 4264
I20260925 10:54:46.946635 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:54:46.947154 4391 heartbeater.cc:347] Connected to a master server at 127.31.62.190:33657
I20260925 10:54:46.947309 31994 external_mini_cluster.cc:1428] Setting key a74ca26879b3de10d4dae05b4cc749a7
I20260925 10:54:46.947517 4391 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:46.948208 4391 heartbeater.cc:511] Master 127.31.62.190:33657 requested a full tablet report, sending...
I20260925 10:54:46.950145 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:33657
--builtin_ntp_servers=127.31.62.148:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260925 10:54:46.950295 4205 ts_manager.cc:194] Registered new tserver with Master: 9160ecec81784d5496606910a48a90b5 (127.31.62.129:37829)
W20260925 10:54:47.274416 4395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:47.274721 4395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:47.274816 4395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:47.282199 4395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:47.282323 4395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:47.282444 4395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:54:47.291960 4395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:47.293316 4395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:47.295193 4395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:47.309178 4401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:47.309190 4403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:47.309245 4400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:47.309988 4395 server_base.cc:1034] running on GCE node
I20260925 10:54:47.310649 4395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:47.311702 4395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:47.312983 4395 hybrid_clock.cc:648] HybridClock initialized: now 1790333687312923 us; error 49 us; skew 500 ppm
I20260925 10:54:47.313410 4395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:54:47.315922 4395 webserver.cc:460] Webserver started at http://127.31.62.130:36137/ using document root <none> and password file <none>
I20260925 10:54:47.316721 4395 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:47.316831 4395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:47.317179 4395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:47.320271 4395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "e5508c1ff6304b769f08fdca2c9b6a9a"
format_stamp: "Formatted at 2026-09-25 10:54:47 on dist-test-slave-nx7s"
server_key: "53a9463dbfdd1b81a8f61ef04c2f6c85"
server_key_iv: "03b710a377301a229c5af7edbf157f46"
server_key_version: "encryptionkey@0"
I20260925 10:54:47.321201 4395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "e5508c1ff6304b769f08fdca2c9b6a9a"
format_stamp: "Formatted at 2026-09-25 10:54:47 on dist-test-slave-nx7s"
server_key: "53a9463dbfdd1b81a8f61ef04c2f6c85"
server_key_iv: "03b710a377301a229c5af7edbf157f46"
server_key_version: "encryptionkey@0"
I20260925 10:54:47.326637 4395 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 10:54:47.330484 4409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:47.332105 4395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:54:47.332295 4395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "e5508c1ff6304b769f08fdca2c9b6a9a"
format_stamp: "Formatted at 2026-09-25 10:54:47 on dist-test-slave-nx7s"
server_key: "53a9463dbfdd1b81a8f61ef04c2f6c85"
server_key_iv: "03b710a377301a229c5af7edbf157f46"
server_key_version: "encryptionkey@0"
I20260925 10:54:47.332521 4395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:47.338585 4395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:47.339442 4395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:47.339696 4395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:47.340649 4395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:47.342140 4395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:47.342237 4395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:47.342351 4395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:47.342423 4395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:47.374414 4395 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:35677
I20260925 10:54:47.374476 4521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:35677 every 8 connection(s)
I20260925 10:54:47.375738 4395 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:54:47.379408 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 4395
I20260925 10:54:47.379601 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:54:47.380019 31994 external_mini_cluster.cc:1428] Setting key 79836c1795f731ab82dc34da660546af
I20260925 10:54:47.382869 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:33657
--builtin_ntp_servers=127.31.62.148:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260925 10:54:47.384603 4393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:47.378510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:46911 (local address 127.31.62.190:33657)
0925 10:54:47.378723 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:47.378728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:47.380343 (+ 1615us) server_negotiation.cc:407] Connection header received
0925 10:54:47.382458 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:47.382465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:47.382529 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:47.382635 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:47.383933 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:47.383939 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:47.383943 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:47.383964 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:47.384002 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:47.384017 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:47.384133 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:47.384270 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 10:54:47.384330 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260925 10:54:47.385663 4524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:47.379089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:33657 (local address 127.31.62.130:46911)
0925 10:54:47.380155 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:47.380209 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:47.382165 (+ 1956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:47.382840 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:47.382848 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:47.382951 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:47.383624 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:54:47.383626 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:47.383666 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:54:47.383757 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:47.384102 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:47.384107 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:47.384112 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 10:54:47.384355 (+ 243us) client_negotiation.cc:240] Negotiation successful
0925 10:54:47.384571 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":308,"threads_started":1}
I20260925 10:54:47.387183 4522 heartbeater.cc:347] Connected to a master server at 127.31.62.190:33657
I20260925 10:54:47.387491 4522 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:47.388084 4522 heartbeater.cc:511] Master 127.31.62.190:33657 requested a full tablet report, sending...
I20260925 10:54:47.389918 4205 ts_manager.cc:194] Registered new tserver with Master: e5508c1ff6304b769f08fdca2c9b6a9a (127.31.62.130:35677)
W20260925 10:54:47.708912 4525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:47.709235 4525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:47.709358 4525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:47.716676 4525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:47.716804 4525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:47.716972 4525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:54:47.726254 4525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:47.727527 4525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:47.729305 4525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:47.742614 4530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:47.742576 4531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:47.742545 4533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:47.743388 4525 server_base.cc:1034] running on GCE node
I20260925 10:54:47.744009 4525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:47.745038 4525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:47.746255 4525 hybrid_clock.cc:648] HybridClock initialized: now 1790333687746201 us; error 47 us; skew 500 ppm
I20260925 10:54:47.746588 4525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:54:47.749042 4525 webserver.cc:460] Webserver started at http://127.31.62.131:40901/ using document root <none> and password file <none>
I20260925 10:54:47.749763 4525 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:47.749850 4525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:47.750164 4525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:47.753270 4525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "f74871d7b7464c45ba14b826d7bd11e5"
format_stamp: "Formatted at 2026-09-25 10:54:47 on dist-test-slave-nx7s"
server_key: "fbc1eab142d603300eaa930f95ab5891"
server_key_iv: "a576d48082c70b602660a2164fffe96f"
server_key_version: "encryptionkey@0"
I20260925 10:54:47.754117 4525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "f74871d7b7464c45ba14b826d7bd11e5"
format_stamp: "Formatted at 2026-09-25 10:54:47 on dist-test-slave-nx7s"
server_key: "fbc1eab142d603300eaa930f95ab5891"
server_key_iv: "a576d48082c70b602660a2164fffe96f"
server_key_version: "encryptionkey@0"
I20260925 10:54:47.759392 4525 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260925 10:54:47.763052 4539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:47.764580 4525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:54:47.764755 4525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "f74871d7b7464c45ba14b826d7bd11e5"
format_stamp: "Formatted at 2026-09-25 10:54:47 on dist-test-slave-nx7s"
server_key: "fbc1eab142d603300eaa930f95ab5891"
server_key_iv: "a576d48082c70b602660a2164fffe96f"
server_key_version: "encryptionkey@0"
I20260925 10:54:47.765021 4525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:47.770910 4525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:47.771720 4525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:47.771963 4525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:47.772926 4525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:47.774364 4525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:47.774446 4525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:47.774526 4525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:47.774578 4525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:47.804874 4525 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:40703
I20260925 10:54:47.804940 4651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:40703 every 8 connection(s)
I20260925 10:54:47.806255 4525 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:54:47.814105 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 4525
I20260925 10:54:47.814328 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:54:47.814865 31994 external_mini_cluster.cc:1428] Setting key d1ebc09b68fc291a2480b925bf8172bb
I20260925 10:54:47.815004 4393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:47.809128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:52275 (local address 127.31.62.190:33657)
0925 10:54:47.809335 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:47.809341 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:47.810936 (+ 1595us) server_negotiation.cc:407] Connection header received
0925 10:54:47.812649 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:47.812653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:47.812711 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:47.812841 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:47.814257 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:47.814263 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:47.814268 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:47.814290 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:47.814333 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:47.814349 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:47.814488 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:47.814660 (+ 172us) server_negotiation.cc:299] Negotiation successful
0925 10:54:47.814730 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 10:54:47.816275 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:47.809738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:33657 (local address 127.31.62.131:52275)
0925 10:54:47.810759 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:47.810799 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:47.812343 (+ 1544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:47.813078 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:47.813091 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:47.813205 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:47.813921 (+ 716us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:54:47.813924 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:47.813964 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:54:47.814055 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:47.814465 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:47.814475 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:47.814481 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 10:54:47.814710 (+ 229us) client_negotiation.cc:240] Negotiation successful
0925 10:54:47.814935 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":299,"threads_started":1}
I20260925 10:54:47.817689 4652 heartbeater.cc:347] Connected to a master server at 127.31.62.190:33657
I20260925 10:54:47.818017 4652 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:47.818607 4652 heartbeater.cc:511] Master 127.31.62.190:33657 requested a full tablet report, sending...
I20260925 10:54:47.820019 4205 ts_manager.cc:194] Registered new tserver with Master: f74871d7b7464c45ba14b826d7bd11e5 (127.31.62.131:40703)
I20260925 10:54:47.830750 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:54:47.837579 4393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:47.835158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49600 (local address 127.31.62.190:33657)
0925 10:54:47.835316 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:47.835320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:47.835766 (+ 446us) server_negotiation.cc:407] Connection header received
0925 10:54:47.835892 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:47.835897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:47.835973 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:47.836095 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:47.837031 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:47.837037 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:47.837042 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:47.837063 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:47.837103 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:47.837120 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:47.837246 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:47.837288 (+ 42us) server_negotiation.cc:299] Negotiation successful
0925 10:54:47.837337 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 10:54:47.841929 4205 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:54:47.849038 4205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:54:47.880227 4664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:47.861859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:40703 (local address 127.0.0.1:40918)
0925 10:54:47.875654 (+ 13795us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:47.875715 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:47.875955 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:47.877768 (+ 1813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:47.877774 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:47.877806 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:47.878731 (+ 925us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:54:47.878735 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:47.878777 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:54:47.878915 (+ 138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:47.879454 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:47.879458 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:47.879463 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 10:54:47.879728 (+ 265us) client_negotiation.cc:240] Negotiation successful
0925 10:54:47.879781 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13531,"thread_start_us":13296,"threads_started":1}
I20260925 10:54:47.880585 4667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:47.862340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40918 (local address 127.31.62.131:40703)
0925 10:54:47.874743 (+ 12403us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:47.874752 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:47.875860 (+ 1108us) server_negotiation.cc:407] Connection header received
0925 10:54:47.876239 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:47.876248 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:47.876450 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:47.876600 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:47.879131 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:47.879139 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:47.879152 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:47.879172 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:47.879230 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:47.879249 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:47.879406 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:47.880103 (+ 697us) server_negotiation.cc:299] Negotiation successful
0925 10:54:47.880210 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12183,"thread_start_us":12423,"threads_started":1}
I20260925 10:54:47.880978 4665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:47.861733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42012 (local address 127.31.62.129:37829)
0925 10:54:47.874717 (+ 12984us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:47.874729 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:47.876009 (+ 1280us) server_negotiation.cc:407] Connection header received
0925 10:54:47.876092 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:47.876101 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:47.876347 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:47.876503 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:47.879110 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:47.879121 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:47.879140 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:47.879161 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:47.879224 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:47.879250 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:47.879377 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:47.880580 (+ 1203us) server_negotiation.cc:299] Negotiation successful
0925 10:54:47.880684 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12775,"thread_start_us":13033,"threads_started":1}
I20260925 10:54:47.881297 4666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:47.861616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:37829 (local address 127.0.0.1:42012)
0925 10:54:47.875654 (+ 14038us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:47.875714 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:47.875896 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:47.877614 (+ 1718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:47.877623 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:47.877730 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:47.878808 (+ 1078us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:54:47.878812 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:47.878830 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:54:47.878915 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:47.879424 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:47.879434 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:47.879441 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 10:54:47.879619 (+ 178us) client_negotiation.cc:240] Negotiation successful
0925 10:54:47.879707 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13741,"thread_start_us":13553,"threads_started":1}
I20260925 10:54:47.883039 4668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:47.876432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:35677 (local address 127.0.0.1:59942)
0925 10:54:47.879103 (+ 2671us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:47.879121 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:47.879207 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:47.880090 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:47.880095 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:47.880120 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:47.881236 (+ 1116us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:54:47.881239 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:47.881255 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:54:47.881305 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:47.882392 (+ 1087us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:47.882397 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:47.882401 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 10:54:47.882547 (+ 146us) client_negotiation.cc:240] Negotiation successful
0925 10:54:47.882597 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2538,"thread_start_us":2366,"threads_started":1}
I20260925 10:54:47.883713 4669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:47.876551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59942 (local address 127.31.62.130:35677)
0925 10:54:47.879478 (+ 2927us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:47.879486 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:47.879514 (+ 28us) server_negotiation.cc:407] Connection header received
0925 10:54:47.879577 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:47.879585 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:47.879822 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:47.879964 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:47.882030 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:47.882044 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:47.882057 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:47.882087 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:47.882143 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:47.882166 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:47.882336 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:47.883315 (+ 979us) server_negotiation.cc:299] Negotiation successful
0925 10:54:47.883409 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2766,"thread_start_us":1914,"threads_started":1}
I20260925 10:54:47.884811 4587 tablet_service.cc:1467] Processing CreateTablet for tablet 99aa5712ac4b4e4eb656a3e6b5018d32 (DEFAULT_TABLE table=test-table [id=f3af6061a3714a5eadcf4337dcb1e50a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:47.884984 4326 tablet_service.cc:1467] Processing CreateTablet for tablet 99aa5712ac4b4e4eb656a3e6b5018d32 (DEFAULT_TABLE table=test-table [id=f3af6061a3714a5eadcf4337dcb1e50a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:47.886314 4587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99aa5712ac4b4e4eb656a3e6b5018d32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:47.886646 4326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99aa5712ac4b4e4eb656a3e6b5018d32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:47.887535 4457 tablet_service.cc:1467] Processing CreateTablet for tablet 99aa5712ac4b4e4eb656a3e6b5018d32 (DEFAULT_TABLE table=test-table [id=f3af6061a3714a5eadcf4337dcb1e50a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:54:47.888864 4457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99aa5712ac4b4e4eb656a3e6b5018d32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:47.897902 4670 tablet_bootstrap.cc:492] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5: Bootstrap starting.
I20260925 10:54:47.901027 4671 tablet_bootstrap.cc:492] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5: Bootstrap starting.
I20260925 10:54:47.902856 4670 tablet_bootstrap.cc:654] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:47.903087 4672 tablet_bootstrap.cc:492] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a: Bootstrap starting.
I20260925 10:54:47.904271 4670 log.cc:824] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:47.905043 4671 tablet_bootstrap.cc:654] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:47.906266 4671 log.cc:824] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:47.906534 4672 tablet_bootstrap.cc:654] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:47.907094 4670 tablet_bootstrap.cc:492] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5: No bootstrap required, opened a new log
I20260925 10:54:47.907382 4670 ts_tablet_manager.cc:1397] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260925 10:54:47.907567 4672 log.cc:824] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:47.909291 4671 tablet_bootstrap.cc:492] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5: No bootstrap required, opened a new log
I20260925 10:54:47.909622 4671 ts_tablet_manager.cc:1397] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260925 10:54:47.910403 4672 tablet_bootstrap.cc:492] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a: No bootstrap required, opened a new log
I20260925 10:54:47.910663 4672 ts_tablet_manager.cc:1397] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260925 10:54:47.916944 4671 raft_consensus.cc:348] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9160ecec81784d5496606910a48a90b5" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37829 } } peers { permanent_uuid: "f74871d7b7464c45ba14b826d7bd11e5" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40703 } } peers { permanent_uuid: "e5508c1ff6304b769f08fdca2c9b6a9a" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35677 } }
I20260925 10:54:47.917484 4671 raft_consensus.cc:374] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:47.917609 4671 raft_consensus.cc:729] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9160ecec81784d5496606910a48a90b5, State: Initialized, Role: FOLLOWER
I20260925 10:54:47.917887 4670 raft_consensus.cc:348] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9160ecec81784d5496606910a48a90b5" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37829 } } peers { permanent_uuid: "f74871d7b7464c45ba14b826d7bd11e5" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40703 } } peers { permanent_uuid: "e5508c1ff6304b769f08fdca2c9b6a9a" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35677 } }
I20260925 10:54:47.918378 4670 raft_consensus.cc:374] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:47.918494 4670 raft_consensus.cc:729] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74871d7b7464c45ba14b826d7bd11e5, State: Initialized, Role: FOLLOWER
I20260925 10:54:47.918435 4671 consensus_queue.cc:260] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9160ecec81784d5496606910a48a90b5" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37829 } } peers { permanent_uuid: "f74871d7b7464c45ba14b826d7bd11e5" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40703 } } peers { permanent_uuid: "e5508c1ff6304b769f08fdca2c9b6a9a" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35677 } }
I20260925 10:54:47.919246 4670 consensus_queue.cc:260] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9160ecec81784d5496606910a48a90b5" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37829 } } peers { permanent_uuid: "f74871d7b7464c45ba14b826d7bd11e5" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40703 } } peers { permanent_uuid: "e5508c1ff6304b769f08fdca2c9b6a9a" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35677 } }
I20260925 10:54:47.920034 4391 heartbeater.cc:503] Master 127.31.62.190:33657 was elected leader, sending a full tablet report...
I20260925 10:54:47.920814 4671 ts_tablet_manager.cc:1428] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5: Time spent starting tablet: real 0.011s user 0.009s sys 0.001s
I20260925 10:54:47.921017 4672 raft_consensus.cc:348] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9160ecec81784d5496606910a48a90b5" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37829 } } peers { permanent_uuid: "f74871d7b7464c45ba14b826d7bd11e5" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40703 } } peers { permanent_uuid: "e5508c1ff6304b769f08fdca2c9b6a9a" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35677 } }
I20260925 10:54:47.921664 4672 raft_consensus.cc:374] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:47.921795 4672 raft_consensus.cc:729] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5508c1ff6304b769f08fdca2c9b6a9a, State: Initialized, Role: FOLLOWER
I20260925 10:54:47.922586 4672 consensus_queue.cc:260] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9160ecec81784d5496606910a48a90b5" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37829 } } peers { permanent_uuid: "f74871d7b7464c45ba14b826d7bd11e5" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40703 } } peers { permanent_uuid: "e5508c1ff6304b769f08fdca2c9b6a9a" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35677 } }
I20260925 10:54:47.922923 4670 ts_tablet_manager.cc:1428] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5: Time spent starting tablet: real 0.015s user 0.009s sys 0.007s
I20260925 10:54:47.923724 4522 heartbeater.cc:503] Master 127.31.62.190:33657 was elected leader, sending a full tablet report...
I20260925 10:54:47.924297 4672 ts_tablet_manager.cc:1428] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
I20260925 10:54:47.925163 4652 heartbeater.cc:503] Master 127.31.62.190:33657 was elected leader, sending a full tablet report...
W20260925 10:54:47.938275 4392 tablet.cc:2369] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:54:47.958042 4678 raft_consensus.cc:482] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:54:47.958356 4678 raft_consensus.cc:504] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9160ecec81784d5496606910a48a90b5" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37829 } } peers { permanent_uuid: "f74871d7b7464c45ba14b826d7bd11e5" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40703 } } peers { permanent_uuid: "e5508c1ff6304b769f08fdca2c9b6a9a" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35677 } }
I20260925 10:54:47.960281 4678 leader_election.cc:288] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9160ecec81784d5496606910a48a90b5 (127.31.62.129:37829), f74871d7b7464c45ba14b826d7bd11e5 (127.31.62.131:40703)
I20260925 10:54:47.963668 4665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:47.960627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:53431 (local address 127.31.62.129:37829)
0925 10:54:47.960824 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:47.960830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:47.961800 (+ 970us) server_negotiation.cc:407] Connection header received
0925 10:54:47.961904 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:47.961909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:47.961960 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:47.962040 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:47.963090 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:47.963096 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:47.963099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:47.963115 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:47.963144 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:47.963155 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:47.963266 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:47.963339 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 10:54:47.963401 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 10:54:47.963682 4667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:47.960773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:54931 (local address 127.31.62.131:40703)
0925 10:54:47.960970 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:47.960976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:47.961732 (+ 756us) server_negotiation.cc:407] Connection header received
0925 10:54:47.961876 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:47.961882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:47.961934 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:47.962012 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:47.963048 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:47.963053 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:47.963058 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:47.963076 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:47.963105 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:47.963117 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:47.963228 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:47.963371 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 10:54:47.963433 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260925 10:54:47.963968 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:47.960673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:40703 (local address 127.31.62.130:54931)
0925 10:54:47.961619 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:47.961635 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:47.961764 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:47.962051 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:47.962055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:47.962086 (+ 31us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:47.962832 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:54:47.962835 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:47.962849 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:54:47.962890 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:47.963260 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:47.963265 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:47.963270 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 10:54:47.963432 (+ 162us) client_negotiation.cc:240] Negotiation successful
0925 10:54:47.963475 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":579,"threads_started":1}
I20260925 10:54:47.964021 4680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:47.960740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:37829 (local address 127.31.62.130:53431)
0925 10:54:47.961689 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:47.961703 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:47.961807 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:47.962069 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:47.962073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:47.962093 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:47.962901 (+ 808us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:54:47.962903 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:47.962915 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:54:47.962948 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:47.963236 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:47.963240 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:47.963245 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 10:54:47.963413 (+ 168us) client_negotiation.cc:240] Negotiation successful
0925 10:54:47.963463 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"thread_start_us":629,"threads_started":1}
I20260925 10:54:47.964950 4607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99aa5712ac4b4e4eb656a3e6b5018d32" candidate_uuid: "e5508c1ff6304b769f08fdca2c9b6a9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74871d7b7464c45ba14b826d7bd11e5" is_pre_election: true
I20260925 10:54:47.964927 4346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99aa5712ac4b4e4eb656a3e6b5018d32" candidate_uuid: "e5508c1ff6304b769f08fdca2c9b6a9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9160ecec81784d5496606910a48a90b5" is_pre_election: true
I20260925 10:54:47.965423 4607 raft_consensus.cc:2454] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5508c1ff6304b769f08fdca2c9b6a9a in term 0.
I20260925 10:54:47.965425 4346 raft_consensus.cc:2454] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5508c1ff6304b769f08fdca2c9b6a9a in term 0.
I20260925 10:54:47.966029 4410 leader_election.cc:302] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5508c1ff6304b769f08fdca2c9b6a9a, f74871d7b7464c45ba14b826d7bd11e5; no voters:
I20260925 10:54:47.966470 4678 raft_consensus.cc:2781] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:54:47.966611 4678 raft_consensus.cc:482] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:54:47.966712 4678 raft_consensus.cc:3037] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:47.969007 4678 raft_consensus.cc:504] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9160ecec81784d5496606910a48a90b5" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37829 } } peers { permanent_uuid: "f74871d7b7464c45ba14b826d7bd11e5" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40703 } } peers { permanent_uuid: "e5508c1ff6304b769f08fdca2c9b6a9a" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35677 } }
I20260925 10:54:47.969587 4678 leader_election.cc:288] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [CANDIDATE]: Term 1 election: Requested vote from peers 9160ecec81784d5496606910a48a90b5 (127.31.62.129:37829), f74871d7b7464c45ba14b826d7bd11e5 (127.31.62.131:40703)
I20260925 10:54:47.970049 4346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99aa5712ac4b4e4eb656a3e6b5018d32" candidate_uuid: "e5508c1ff6304b769f08fdca2c9b6a9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9160ecec81784d5496606910a48a90b5"
I20260925 10:54:47.970060 4607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99aa5712ac4b4e4eb656a3e6b5018d32" candidate_uuid: "e5508c1ff6304b769f08fdca2c9b6a9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74871d7b7464c45ba14b826d7bd11e5"
I20260925 10:54:47.970237 4346 raft_consensus.cc:3037] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:47.970244 4607 raft_consensus.cc:3037] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:47.972409 4346 raft_consensus.cc:2454] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5508c1ff6304b769f08fdca2c9b6a9a in term 1.
I20260925 10:54:47.972486 4607 raft_consensus.cc:2454] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5508c1ff6304b769f08fdca2c9b6a9a in term 1.
I20260925 10:54:47.972823 4412 leader_election.cc:302] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9160ecec81784d5496606910a48a90b5, e5508c1ff6304b769f08fdca2c9b6a9a; no voters:
I20260925 10:54:47.973186 4678 raft_consensus.cc:2781] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:47.973590 4678 raft_consensus.cc:686] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [term 1 LEADER]: Becoming Leader. State: Replica: e5508c1ff6304b769f08fdca2c9b6a9a, State: Running, Role: LEADER
I20260925 10:54:47.974017 4678 consensus_queue.cc:237] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9160ecec81784d5496606910a48a90b5" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37829 } } peers { permanent_uuid: "f74871d7b7464c45ba14b826d7bd11e5" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40703 } } peers { permanent_uuid: "e5508c1ff6304b769f08fdca2c9b6a9a" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35677 } }
I20260925 10:54:47.978655 4205 catalog_manager.cc:5495] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a reported cstate change: term changed from 0 to 1, leader changed from <none> to e5508c1ff6304b769f08fdca2c9b6a9a (127.31.62.130). New cstate: current_term: 1 leader_uuid: "e5508c1ff6304b769f08fdca2c9b6a9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9160ecec81784d5496606910a48a90b5" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f74871d7b7464c45ba14b826d7bd11e5" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5508c1ff6304b769f08fdca2c9b6a9a" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35677 } health_report { overall_health: HEALTHY } } }
I20260925 10:54:48.012765 4669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:48.010496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59956 (local address 127.31.62.130:35677)
0925 10:54:48.010672 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:48.010677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:48.010739 (+ 62us) server_negotiation.cc:407] Connection header received
0925 10:54:48.011034 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:48.011040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:48.011216 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:48.011308 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:48.012215 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:48.012220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:48.012223 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:48.012238 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:48.012270 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:48.012282 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:48.012387 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:48.012439 (+ 52us) server_negotiation.cc:299] Negotiation successful
0925 10:54:48.012505 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 10:54:48.021148 4346 raft_consensus.cc:1261] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5 [term 1 FOLLOWER]: Refusing update from remote peer e5508c1ff6304b769f08fdca2c9b6a9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:48.021152 4607 raft_consensus.cc:1261] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5 [term 1 FOLLOWER]: Refusing update from remote peer e5508c1ff6304b769f08fdca2c9b6a9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:54:48.022109 4678 consensus_queue.cc:1035] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f74871d7b7464c45ba14b826d7bd11e5" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:48.022388 4681 consensus_queue.cc:1035] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9160ecec81784d5496606910a48a90b5" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:54:48.048200 4205 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:48.048437 4205 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:54:48.051987 4205 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b06530e55a7a4dd8bd171401ef899894: Sending DeleteTablet for 3 replicas of tablet 99aa5712ac4b4e4eb656a3e6b5018d32
I20260925 10:54:48.053467 4587 tablet_service.cc:1514] Processing DeleteTablet for tablet 99aa5712ac4b4e4eb656a3e6b5018d32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:48 UTC) from {username='slave'} at 127.0.0.1:40918
I20260925 10:54:48.053464 4326 tablet_service.cc:1514] Processing DeleteTablet for tablet 99aa5712ac4b4e4eb656a3e6b5018d32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:48 UTC) from {username='slave'} at 127.0.0.1:42012
I20260925 10:54:48.054498 4457 tablet_service.cc:1514] Processing DeleteTablet for tablet 99aa5712ac4b4e4eb656a3e6b5018d32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:54:48 UTC) from {username='slave'} at 127.0.0.1:59942
I20260925 10:54:48.055138 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 4264
I20260925 10:54:48.064332 4695 tablet_replica.cc:330] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a: stopping tablet replica
I20260925 10:54:48.064960 4695 raft_consensus.cc:2229] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:54:48.065840 4695 raft_consensus.cc:2258] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:48.071091 4695 ts_tablet_manager.cc:1905] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:48.071228 4665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:48.056895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42028 (local address 127.31.62.129:37829)
0925 10:54:48.057100 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:48.057108 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:48.068266 (+ 11158us) server_negotiation.cc:407] Connection header received
0925 10:54:48.068687 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:48.068695 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:48.068827 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:48.068945 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:48.070354 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:48.070361 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:48.070366 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:48.070389 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:48.070429 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:48.070447 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:48.070564 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:48.070834 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 10:54:48.070904 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 10:54:48.072567 4386 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:48.081135 4693 tablet_replica.cc:330] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5: stopping tablet replica
I20260925 10:54:48.082728 4693 raft_consensus.cc:2229] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:54:48.084365 4695 ts_tablet_manager.cc:1918] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:54:48.084610 4695 log.cc:1196] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/99aa5712ac4b4e4eb656a3e6b5018d32
I20260925 10:54:48.085314 4695 ts_tablet_manager.cc:1939] T 99aa5712ac4b4e4eb656a3e6b5018d32 P e5508c1ff6304b769f08fdca2c9b6a9a: Deleting consensus metadata
I20260925 10:54:48.085412 4693 raft_consensus.cc:2258] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:54:48.087981 4189 catalog_manager.cc:4841] TS e5508c1ff6304b769f08fdca2c9b6a9a (127.31.62.130:35677): tablet 99aa5712ac4b4e4eb656a3e6b5018d32 (table test-table [id=f3af6061a3714a5eadcf4337dcb1e50a]) successfully deleted
I20260925 10:54:48.088307 4693 ts_tablet_manager.cc:1905] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:54:48.095588 4693 ts_tablet_manager.cc:1918] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:54:48.095810 4693 log.cc:1196] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/99aa5712ac4b4e4eb656a3e6b5018d32
I20260925 10:54:48.096379 4693 ts_tablet_manager.cc:1939] T 99aa5712ac4b4e4eb656a3e6b5018d32 P f74871d7b7464c45ba14b826d7bd11e5: Deleting consensus metadata
I20260925 10:54:48.098361 4189 catalog_manager.cc:4841] TS f74871d7b7464c45ba14b826d7bd11e5 (127.31.62.131:40703): tablet 99aa5712ac4b4e4eb656a3e6b5018d32 (table test-table [id=f3af6061a3714a5eadcf4337dcb1e50a]) successfully deleted
I20260925 10:54:48.175709 4694 tablet_replica.cc:330] T 99aa5712ac4b4e4eb656a3e6b5018d32 P 9160ecec81784d5496606910a48a90b5: stopping tablet replica
I20260925 10:54:48.176110 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 4264
I20260925 10:54:48.197903 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 4395
W20260925 10:54:48.198097 4191 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.62.129:37829 (error 108)
I20260925 10:54:48.201087 4668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:48.200601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:37829 (local address 127.0.0.1:42032)
0925 10:54:48.200755 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:48.200908 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:37829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 10:54:48.201359 4669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:48.199055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59958 (local address 127.31.62.130:35677)
0925 10:54:48.199247 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:48.199254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:48.199277 (+ 23us) server_negotiation.cc:407] Connection header received
0925 10:54:48.199375 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:48.199381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:48.199454 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:48.199568 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:48.200543 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:48.200549 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:48.200554 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:48.200573 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:48.200611 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:48.200627 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:48.200763 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:48.201015 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 10:54:48.201077 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260925 10:54:48.201444 4191 catalog_manager.cc:4584] TS 9160ecec81784d5496606910a48a90b5 (127.31.62.129:37829): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 99aa5712ac4b4e4eb656a3e6b5018d32: Network error: Client connection negotiation failed: client connection to 127.31.62.129:37829: connect: Connection refused (error 111)
I20260925 10:54:48.202104 4517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:48.255117 4668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:48.254624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:37829 (local address 127.0.0.1:42040)
0925 10:54:48.254859 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:48.254956 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:37829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260925 10:54:48.256253 4668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:48.255883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:37829 (local address 127.0.0.1:42044)
0925 10:54:48.256032 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:48.256107 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:37829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260925 10:54:48.309368 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 4395
I20260925 10:54:48.316994 4668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:48.316454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:37829 (local address 127.0.0.1:42056)
0925 10:54:48.316679 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:48.316804 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:37829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260925 10:54:48.318032 4668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:48.317703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:37829 (local address 127.0.0.1:42072)
0925 10:54:48.317842 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:48.317914 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:37829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260925 10:54:48.324033 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 4525
I20260925 10:54:48.326840 4667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:48.324817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40934 (local address 127.31.62.131:40703)
0925 10:54:48.324999 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:48.325004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:48.325025 (+ 21us) server_negotiation.cc:407] Connection header received
0925 10:54:48.325261 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:48.325267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:48.325349 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:48.325430 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:48.326253 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:48.326259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:48.326264 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:48.326281 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:48.326309 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:48.326321 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:48.326430 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:48.326509 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 10:54:48.326568 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 10:54:48.327515 4647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:48.399066 4668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:48.398496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:37829 (local address 127.0.0.1:42078)
0925 10:54:48.398741 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:48.398893 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:37829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260925 10:54:48.400166 4668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:48.399787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:37829 (local address 127.0.0.1:42094)
0925 10:54:48.399982 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:48.400052 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:37829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260925 10:54:48.414552 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 4525
I20260925 10:54:48.428663 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 4174
I20260925 10:54:48.430181 4235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:48.533211 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 4174
2026-09-25T10:54:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2602 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260925 10:54:48.569260 31994 mini_oidc.cc:152] Starting JWKS server
I20260925 10:54:48.571846 31994 webserver.cc:460] Webserver started at https://127.0.0.1:41931/ using document root <none> and password file <none>
I20260925 10:54:48.572139 31994 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 10:54:48.572814 31994 webserver.cc:460] Webserver started at http://127.0.0.1:34305/ using document root <none> and password file <none>
2026-09-25T10:54:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:48Z Disabled control of system clock
I20260925 10:54:48.598416 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:45185
--webserver_interface=127.31.62.190
--webserver_port=0
--builtin_ntp_servers=127.31.62.148:40325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:45185
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41931/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/testchainca.cert with env {}
W20260925 10:54:48.916177 4707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:48.916473 4707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:48.916603 4707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:48.923944 4707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:48.924067 4707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:48.924113 4707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:48.924190 4707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:48.924244 4707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 10:54:48.924322 4707 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 10:54:48.924374 4707 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41931/jwks.json
I20260925 10:54:48.934590 4707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:40325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:45185
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:45185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41931/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:48.936034 4707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:48.937426 4707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:48.949635 4713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:48.949745 4715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:48.949777 4712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:48.950565 4707 server_base.cc:1034] running on GCE node
I20260925 10:54:48.951406 4707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:48.953001 4707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:48.954233 4707 hybrid_clock.cc:648] HybridClock initialized: now 1790333688954187 us; error 34 us; skew 500 ppm
I20260925 10:54:48.954619 4707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:54:48.956947 4707 webserver.cc:460] Webserver started at http://127.31.62.190:42175/ using document root <none> and password file <none>
I20260925 10:54:48.957656 4707 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:48.957763 4707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:48.958060 4707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:48.961162 4707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "f415dcff350c4b2fb9d204585f3ce82d"
format_stamp: "Formatted at 2026-09-25 10:54:48 on dist-test-slave-nx7s"
server_key: "0ae409db3e9c25d5294cd295fc168994"
server_key_iv: "20b59ef130fddd87302f66d4028cda7f"
server_key_version: "encryptionkey@0"
I20260925 10:54:48.961999 4707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "f415dcff350c4b2fb9d204585f3ce82d"
format_stamp: "Formatted at 2026-09-25 10:54:48 on dist-test-slave-nx7s"
server_key: "0ae409db3e9c25d5294cd295fc168994"
server_key_iv: "20b59ef130fddd87302f66d4028cda7f"
server_key_version: "encryptionkey@0"
I20260925 10:54:48.967352 4707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:54:48.970909 4721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:48.972366 4707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:54:48.972543 4707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "f415dcff350c4b2fb9d204585f3ce82d"
format_stamp: "Formatted at 2026-09-25 10:54:48 on dist-test-slave-nx7s"
server_key: "0ae409db3e9c25d5294cd295fc168994"
server_key_iv: "20b59ef130fddd87302f66d4028cda7f"
server_key_version: "encryptionkey@0"
I20260925 10:54:48.972755 4707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:48.993379 4707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:48.994288 4707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:48.994598 4707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:49.011965 4707 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:45185
I20260925 10:54:49.011983 4772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:45185 every 8 connection(s)
I20260925 10:54:49.013396 4707 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:49.016196 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 4707
I20260925 10:54:49.016403 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:49.017215 31994 external_mini_cluster.cc:1428] Setting key 20ce23f114b60fff0366f8bfd63ca3be
I20260925 10:54:49.017985 4773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:49.030745 4773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d: Bootstrap starting.
I20260925 10:54:49.031560 4776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:49.019323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59790 (local address 127.31.62.190:45185)
0925 10:54:49.020386 (+ 1063us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:49.020398 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:49.020441 (+ 43us) server_negotiation.cc:407] Connection header received
0925 10:54:49.022793 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:49.022819 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:49.023400 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:49.023815 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:49.024437 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:49.025818 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:49.027174 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:49.027878 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:49.028058 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:49.029215 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:49.029229 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:49.029246 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:49.029787 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:49.029870 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:49.029892 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:49.029991 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:49.030210 (+ 219us) server_negotiation.cc:299] Negotiation successful
0925 10:54:49.030531 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":355,"threads_started":1}
I20260925 10:54:49.034034 4773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:49.035131 4773 log.cc:824] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:49.035403 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:54:49.038002 4773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d: No bootstrap required, opened a new log
I20260925 10:54:49.045431 4773 raft_consensus.cc:348] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f415dcff350c4b2fb9d204585f3ce82d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45185 } }
I20260925 10:54:49.045827 4773 raft_consensus.cc:374] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:49.045928 4773 raft_consensus.cc:729] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f415dcff350c4b2fb9d204585f3ce82d, State: Initialized, Role: FOLLOWER
I20260925 10:54:49.046500 4773 consensus_queue.cc:260] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f415dcff350c4b2fb9d204585f3ce82d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45185 } }
I20260925 10:54:49.046716 4773 raft_consensus.cc:388] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:49.046842 4773 raft_consensus.cc:482] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:49.046988 4773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:49.049553 4773 raft_consensus.cc:504] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f415dcff350c4b2fb9d204585f3ce82d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45185 } }
I20260925 10:54:49.050088 4773 leader_election.cc:302] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f415dcff350c4b2fb9d204585f3ce82d; no voters:
I20260925 10:54:49.050668 4773 leader_election.cc:288] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:49.050806 4778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:49.051626 4778 raft_consensus.cc:686] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [term 1 LEADER]: Becoming Leader. State: Replica: f415dcff350c4b2fb9d204585f3ce82d, State: Running, Role: LEADER
I20260925 10:54:49.052145 4778 consensus_queue.cc:237] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f415dcff350c4b2fb9d204585f3ce82d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45185 } }
I20260925 10:54:49.052742 4773 sys_catalog.cc:564] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:49.054979 4779 sys_catalog.cc:455] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f415dcff350c4b2fb9d204585f3ce82d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f415dcff350c4b2fb9d204585f3ce82d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45185 } } }
I20260925 10:54:49.055418 4779 sys_catalog.cc:458] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:49.055006 4780 sys_catalog.cc:455] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f415dcff350c4b2fb9d204585f3ce82d. Latest consensus state: current_term: 1 leader_uuid: "f415dcff350c4b2fb9d204585f3ce82d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f415dcff350c4b2fb9d204585f3ce82d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45185 } } }
I20260925 10:54:49.057317 4780 sys_catalog.cc:458] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:49.057914 4784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:49.063028 4784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:49.072891 4784 catalog_manager.cc:1347] Generated new cluster ID: 486ae62a97f5485e81c2459f6a956817
I20260925 10:54:49.073026 4784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:49.082007 4784 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:49.083390 4784 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:49.097002 4784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f415dcff350c4b2fb9d204585f3ce82d: Generated new TSK 0
I20260925 10:54:49.098024 4784 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:54:49.148648 4776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:49.119381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59792 (local address 127.31.62.190:45185)
0925 10:54:49.119656 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:49.119663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:49.120105 (+ 442us) server_negotiation.cc:407] Connection header received
0925 10:54:49.120487 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:49.120492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:49.120572 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:49.120703 (+ 131us) server_negotiation.cc:226] Negotiated authn=JWT
0925 10:54:49.121368 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:49.122133 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:49.124247 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:49.124682 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:49.124798 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:49.125550 (+ 752us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 10:54:49.148108 (+ 22558us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41931/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 10:54:49.148419 (+ 311us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59792: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41931/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":83}
W20260925 10:54:49.148818 4776 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59792: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41931/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
E20260925 10:54:49.149199 4803 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260925 10:54:49.149345 4803 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260925 10:54:49.150697 4802 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:54:49.119253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:45185 (local address 127.0.0.1:59792)
0925 10:54:49.119984 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:49.120009 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:49.120208 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:49.120760 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:49.120766 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:49.120780 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0925 10:54:49.121217 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:49.121229 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:49.122261 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:49.122267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:49.124077 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:49.124094 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:49.124899 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:49.124907 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:49.125094 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:49.125195 (+ 101us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 10:54:49.148446 (+ 23251us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41931/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 10:54:49.148828 (+ 382us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.62.190:45185: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41931/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":350,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.31.62.190:45185: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41931/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260925 10:54:49.152437 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 4707
I20260925 10:54:49.153781 4768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:49.247174 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 4707
2026-09-25T10:54:49Z chronyd exiting
I20260925 10:54:49.293900 31994 test_util.cc:175] -----------------------------------------------
I20260925 10:54:49.294040 31994 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790333647382755-31994-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (729 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260925 10:54:49.298146 31994 mini_oidc.cc:152] Starting JWKS server
I20260925 10:54:49.299923 31994 webserver.cc:460] Webserver started at https://127.0.0.1:32993/ using document root <none> and password file <none>
I20260925 10:54:49.300179 31994 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 10:54:49.300788 31994 webserver.cc:460] Webserver started at http://127.0.0.1:34203/ using document root <none> and password file <none>
2026-09-25T10:54:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:49Z Disabled control of system clock
I20260925 10:54:49.326553 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:36869
--webserver_interface=127.31.62.190
--webserver_port=0
--builtin_ntp_servers=127.31.62.148:40547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:36869
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32993/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/testchainca.cert with env {}
W20260925 10:54:49.647996 4810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:49.648334 4810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:49.648474 4810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:49.655854 4810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:49.655988 4810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:49.656060 4810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:49.656136 4810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:49.656204 4810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 10:54:49.656291 4810 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 10:54:49.656344 4810 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32993/jwks.json
I20260925 10:54:49.666497 4810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:40547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:36869
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:36869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32993/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:49.667845 4810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:49.669205 4810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:49.680963 4815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:49.681072 4818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:49.681094 4816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:49.681819 4810 server_base.cc:1034] running on GCE node
I20260925 10:54:49.682665 4810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:49.684128 4810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:49.685340 4810 hybrid_clock.cc:648] HybridClock initialized: now 1790333689685287 us; error 49 us; skew 500 ppm
I20260925 10:54:49.685676 4810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:54:49.688015 4810 webserver.cc:460] Webserver started at http://127.31.62.190:45173/ using document root <none> and password file <none>
I20260925 10:54:49.688733 4810 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:49.688838 4810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:49.689186 4810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:49.692130 4810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "bb372df1b4bf4a78ba9c5e103deedff2"
format_stamp: "Formatted at 2026-09-25 10:54:49 on dist-test-slave-nx7s"
server_key: "d994090c2718745624376ef150a5c90f"
server_key_iv: "159d9756f915424775c6b64e095c4077"
server_key_version: "encryptionkey@0"
I20260925 10:54:49.693027 4810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "bb372df1b4bf4a78ba9c5e103deedff2"
format_stamp: "Formatted at 2026-09-25 10:54:49 on dist-test-slave-nx7s"
server_key: "d994090c2718745624376ef150a5c90f"
server_key_iv: "159d9756f915424775c6b64e095c4077"
server_key_version: "encryptionkey@0"
I20260925 10:54:49.698431 4810 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 10:54:49.702185 4824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:49.703704 4810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:54:49.703872 4810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "bb372df1b4bf4a78ba9c5e103deedff2"
format_stamp: "Formatted at 2026-09-25 10:54:49 on dist-test-slave-nx7s"
server_key: "d994090c2718745624376ef150a5c90f"
server_key_iv: "159d9756f915424775c6b64e095c4077"
server_key_version: "encryptionkey@0"
I20260925 10:54:49.704090 4810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:49.721019 4810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:49.721894 4810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:49.722182 4810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:49.739494 4810 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:36869
I20260925 10:54:49.739516 4875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:36869 every 8 connection(s)
I20260925 10:54:49.740824 4810 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:49.744656 4876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:49.744982 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 4810
I20260925 10:54:49.745172 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:49.745806 31994 external_mini_cluster.cc:1428] Setting key f3be23260d325e7c0e1d44db7a8fe325
I20260925 10:54:49.757707 4876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2: Bootstrap starting.
I20260925 10:54:49.759768 4879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:49.747539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46204 (local address 127.31.62.190:36869)
0925 10:54:49.748593 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:49.748605 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:49.748649 (+ 44us) server_negotiation.cc:407] Connection header received
0925 10:54:49.750331 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:49.750354 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:49.750837 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:49.751184 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:49.751712 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:49.753074 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:49.754864 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:49.755612 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:49.755745 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:49.756992 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:49.757001 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:49.757017 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:49.757616 (+ 599us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:49.757707 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:49.757729 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:49.757817 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:49.758093 (+ 276us) server_negotiation.cc:299] Negotiation successful
0925 10:54:49.758443 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":385,"threads_started":1}
I20260925 10:54:49.761350 4876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:49.762452 4876 log.cc:824] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:49.763808 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:54:49.765882 4876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2: No bootstrap required, opened a new log
I20260925 10:54:49.773607 4876 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb372df1b4bf4a78ba9c5e103deedff2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36869 } }
I20260925 10:54:49.773993 4876 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:49.774067 4876 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb372df1b4bf4a78ba9c5e103deedff2, State: Initialized, Role: FOLLOWER
I20260925 10:54:49.774591 4876 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb372df1b4bf4a78ba9c5e103deedff2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36869 } }
I20260925 10:54:49.774806 4876 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:49.774899 4876 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:49.775022 4876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:49.777379 4876 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb372df1b4bf4a78ba9c5e103deedff2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36869 } }
I20260925 10:54:49.777920 4876 leader_election.cc:302] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb372df1b4bf4a78ba9c5e103deedff2; no voters:
I20260925 10:54:49.778462 4876 leader_election.cc:288] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:49.778636 4881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:49.779363 4881 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [term 1 LEADER]: Becoming Leader. State: Replica: bb372df1b4bf4a78ba9c5e103deedff2, State: Running, Role: LEADER
I20260925 10:54:49.779914 4881 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb372df1b4bf4a78ba9c5e103deedff2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36869 } }
I20260925 10:54:49.780514 4876 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:49.783605 4882 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb372df1b4bf4a78ba9c5e103deedff2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb372df1b4bf4a78ba9c5e103deedff2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36869 } } }
I20260925 10:54:49.783629 4883 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb372df1b4bf4a78ba9c5e103deedff2. Latest consensus state: current_term: 1 leader_uuid: "bb372df1b4bf4a78ba9c5e103deedff2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb372df1b4bf4a78ba9c5e103deedff2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36869 } } }
I20260925 10:54:49.784050 4883 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:49.784050 4882 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:49.785068 4887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:49.792807 4887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:49.802826 4887 catalog_manager.cc:1347] Generated new cluster ID: eb7b33f36824439e8228a1c7fbc1bfd0
I20260925 10:54:49.802955 4887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:49.813620 4887 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:49.814950 4887 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:49.835073 4887 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb372df1b4bf4a78ba9c5e103deedff2: Generated new TSK 0
I20260925 10:54:49.835961 4887 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 10:54:49.875802 4906 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260925 10:54:49.875921 4906 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 10:54:49.877221 4879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:49.856560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46206 (local address 127.31.62.190:36869)
0925 10:54:49.856758 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:49.856764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:49.857078 (+ 314us) server_negotiation.cc:407] Connection header received
0925 10:54:49.857337 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:49.857343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:49.857443 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:49.857607 (+ 164us) server_negotiation.cc:226] Negotiated authn=JWT
0925 10:54:49.858034 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:49.859153 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:49.860611 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:49.861350 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:49.861476 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:49.861966 (+ 490us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 10:54:49.876485 (+ 14519us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32993/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 10:54:49.876917 (+ 432us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46206: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32993/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":72}
W20260925 10:54:49.877336 4905 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:54:49.856366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36869 (local address 127.0.0.1:46206)
0925 10:54:49.856986 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:49.857002 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:49.857128 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:49.857630 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:49.857633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:49.857643 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0925 10:54:49.857889 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:49.857897 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:49.859316 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:49.859321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:49.860472 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:49.860484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:49.861506 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:49.861511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:49.861643 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:49.861679 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 10:54:49.876747 (+ 15068us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32993/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 10:54:49.877029 (+ 282us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.62.190:36869: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32993/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":319,"threads_started":1}
W20260925 10:54:49.877345 4879 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46206: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32993/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260925 10:54:49.878360 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 4810
I20260925 10:54:49.879483 4871 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:49.972899 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 4810
2026-09-25T10:54:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (725 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260925 10:54:50.023838 31994 mini_oidc.cc:152] Starting JWKS server
I20260925 10:54:50.025772 31994 webserver.cc:460] Webserver started at https://127.0.0.1:34691/ using document root <none> and password file <none>
I20260925 10:54:50.026023 31994 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 10:54:50.026692 31994 webserver.cc:460] Webserver started at http://127.0.0.1:46477/ using document root <none> and password file <none>
2026-09-25T10:54:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:50Z Disabled control of system clock
I20260925 10:54:50.052129 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:46573
--webserver_interface=127.31.62.190
--webserver_port=0
--builtin_ntp_servers=127.31.62.148:36199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:46573
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34691/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 10:54:50.377609 4913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:50.377905 4913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:50.378002 4913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:50.385455 4913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:50.385582 4913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:50.385617 4913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:50.385664 4913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:50.385707 4913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 10:54:50.385772 4913 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 10:54:50.385812 4913 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34691/jwks.json
I20260925 10:54:50.395218 4913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:36199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:46573
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:46573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34691/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:50.396498 4913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:50.398365 4913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:50.410799 4919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:50.410943 4921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:50.410846 4918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:50.412642 4913 server_base.cc:1034] running on GCE node
I20260925 10:54:50.413856 4913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:50.415836 4913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:50.417104 4913 hybrid_clock.cc:648] HybridClock initialized: now 1790333690417039 us; error 58 us; skew 500 ppm
I20260925 10:54:50.417554 4913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:54:50.425575 4913 webserver.cc:460] Webserver started at http://127.31.62.190:40583/ using document root <none> and password file <none>
I20260925 10:54:50.426443 4913 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:50.426563 4913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:50.426927 4913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:50.430225 4913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "b69c2f3e866140f5b2eb652dadd16113"
format_stamp: "Formatted at 2026-09-25 10:54:50 on dist-test-slave-nx7s"
server_key: "ea355ba9ad22cdcadff0bea2e278a9d7"
server_key_iv: "ce3ea75ff3f0e8e41411df3200ea9369"
server_key_version: "encryptionkey@0"
I20260925 10:54:50.431150 4913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "b69c2f3e866140f5b2eb652dadd16113"
format_stamp: "Formatted at 2026-09-25 10:54:50 on dist-test-slave-nx7s"
server_key: "ea355ba9ad22cdcadff0bea2e278a9d7"
server_key_iv: "ce3ea75ff3f0e8e41411df3200ea9369"
server_key_version: "encryptionkey@0"
I20260925 10:54:50.437583 4913 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.002s
I20260925 10:54:50.441872 4927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:50.443814 4913 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260925 10:54:50.444021 4913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "b69c2f3e866140f5b2eb652dadd16113"
format_stamp: "Formatted at 2026-09-25 10:54:50 on dist-test-slave-nx7s"
server_key: "ea355ba9ad22cdcadff0bea2e278a9d7"
server_key_iv: "ce3ea75ff3f0e8e41411df3200ea9369"
server_key_version: "encryptionkey@0"
I20260925 10:54:50.444267 4913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:50.473935 4913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:50.474887 4913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:50.475219 4913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:50.494163 4978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:46573 every 8 connection(s)
I20260925 10:54:50.494158 4913 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:46573
I20260925 10:54:50.495757 4913 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:50.499758 4979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:50.501888 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 4913
I20260925 10:54:50.502069 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:50.502789 31994 external_mini_cluster.cc:1428] Setting key c01f71838708e7e0f5da9488c85283fd
I20260925 10:54:50.511785 4979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113: Bootstrap starting.
I20260925 10:54:50.515276 4979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:50.516492 4979 log.cc:824] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:50.517616 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:50.504951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60398 (local address 127.31.62.190:46573)
0925 10:54:50.506197 (+ 1246us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:50.506211 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:50.506262 (+ 51us) server_negotiation.cc:407] Connection header received
0925 10:54:50.507968 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:50.507993 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:50.508561 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:50.509080 (+ 519us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:50.509746 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:50.511042 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:50.512435 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:50.513335 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:50.513527 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:50.514745 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:50.514753 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:50.514802 (+ 49us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:50.515389 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:50.515516 (+ 127us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:50.515540 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:50.515670 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:50.515919 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 10:54:50.516262 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":440,"threads_started":1}
I20260925 10:54:50.519876 4979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113: No bootstrap required, opened a new log
I20260925 10:54:50.522138 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:54:50.528339 4979 raft_consensus.cc:348] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69c2f3e866140f5b2eb652dadd16113" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 46573 } }
I20260925 10:54:50.528753 4979 raft_consensus.cc:374] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:50.528877 4979 raft_consensus.cc:729] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b69c2f3e866140f5b2eb652dadd16113, State: Initialized, Role: FOLLOWER
I20260925 10:54:50.529516 4979 consensus_queue.cc:260] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69c2f3e866140f5b2eb652dadd16113" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 46573 } }
I20260925 10:54:50.529790 4979 raft_consensus.cc:388] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:50.529943 4979 raft_consensus.cc:482] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:50.530112 4979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:50.532372 4979 raft_consensus.cc:504] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69c2f3e866140f5b2eb652dadd16113" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 46573 } }
I20260925 10:54:50.532912 4979 leader_election.cc:302] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b69c2f3e866140f5b2eb652dadd16113; no voters:
I20260925 10:54:50.533493 4979 leader_election.cc:288] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:50.533638 4984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:50.534693 4984 raft_consensus.cc:686] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [term 1 LEADER]: Becoming Leader. State: Replica: b69c2f3e866140f5b2eb652dadd16113, State: Running, Role: LEADER
I20260925 10:54:50.535305 4984 consensus_queue.cc:237] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69c2f3e866140f5b2eb652dadd16113" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 46573 } }
I20260925 10:54:50.535673 4979 sys_catalog.cc:564] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:50.538259 4986 sys_catalog.cc:455] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b69c2f3e866140f5b2eb652dadd16113. Latest consensus state: current_term: 1 leader_uuid: "b69c2f3e866140f5b2eb652dadd16113" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69c2f3e866140f5b2eb652dadd16113" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 46573 } } }
I20260925 10:54:50.538300 4985 sys_catalog.cc:455] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b69c2f3e866140f5b2eb652dadd16113" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69c2f3e866140f5b2eb652dadd16113" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 46573 } } }
I20260925 10:54:50.538739 4985 sys_catalog.cc:458] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:50.538741 4986 sys_catalog.cc:458] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:50.540601 4990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:50.546139 4990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:50.556632 4990 catalog_manager.cc:1347] Generated new cluster ID: 2e6f6db2c1bb491f8231d0891ee071fe
I20260925 10:54:50.556766 4990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:50.568977 4990 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:50.570300 4990 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:50.586802 4990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b69c2f3e866140f5b2eb652dadd16113: Generated new TSK 0
I20260925 10:54:50.587728 4990 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:54:50.607466 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:50.598533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60400 (local address 127.31.62.190:46573)
0925 10:54:50.598729 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:50.598735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:50.599065 (+ 330us) server_negotiation.cc:407] Connection header received
0925 10:54:50.599282 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:50.599288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:50.599389 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:50.599562 (+ 173us) server_negotiation.cc:226] Negotiated authn=JWT
0925 10:54:50.600041 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:50.601223 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:50.602709 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:50.603397 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:50.603529 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:50.604235 (+ 706us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 10:54:50.606817 (+ 2582us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34691/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 10:54:50.607153 (+ 336us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60400: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34691/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":71}
W20260925 10:54:50.607604 4982 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60400: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34691/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260925 10:54:50.609433 5008 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:54:50.598400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:46573 (local address 127.0.0.1:60400)
0925 10:54:50.598967 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:50.598982 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:50.599107 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:50.599617 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:50.599621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:50.599633 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0925 10:54:50.599887 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:50.599895 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:50.601402 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:50.601407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:50.602539 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:50.602562 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:50.603653 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:50.603659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:50.603806 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:50.603845 (+ 39us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 10:54:50.607051 (+ 3206us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34691/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 10:54:50.608944 (+ 1893us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.62.190:46573: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34691/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":1517,"thread_start_us":297,"threads_started":1}
I20260925 10:54:50.610473 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 4913
I20260925 10:54:50.611691 4974 generic_service.cc:196] Checking for leaks (request via RPC)
E20260925 10:54:50.621968 5009 webserver.cc:533] Webserver: sslize failed:
E20260925 10:54:50.622138 5009 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 10:54:50.704953 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 4913
2026-09-25T10:54:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (732 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-25T10:54:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:50Z Disabled control of system clock
I20260925 10:54:50.782847 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:34863
--webserver_interface=127.31.62.190
--webserver_port=0
--builtin_ntp_servers=127.31.62.148:44877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:34863
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/testchainca.cert with env {}
W20260925 10:54:51.115403 5014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:51.115718 5014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:51.115803 5014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:51.123168 5014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:51.123334 5014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:51.123383 5014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:51.123441 5014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:51.123484 5014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 10:54:51.123543 5014 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 10:54:51.123584 5014 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260925 10:54:51.133697 5014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:34863
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:34863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:51.134977 5014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:51.136209 5014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:51.148234 5019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:51.148257 5022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:51.148195 5020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:51.149103 5014 server_base.cc:1034] running on GCE node
I20260925 10:54:51.149947 5014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:51.151446 5014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:51.152707 5014 hybrid_clock.cc:648] HybridClock initialized: now 1790333691152624 us; error 60 us; skew 500 ppm
I20260925 10:54:51.153131 5014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:54:51.155572 5014 webserver.cc:460] Webserver started at http://127.31.62.190:39091/ using document root <none> and password file <none>
I20260925 10:54:51.156286 5014 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:51.156390 5014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:51.156669 5014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:51.159698 5014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "23b163d185db4b66b3ab34fe5314518d"
format_stamp: "Formatted at 2026-09-25 10:54:51 on dist-test-slave-nx7s"
server_key: "f679a3279e38d308fe084f44cc9cdb7e"
server_key_iv: "bfe474b5910a4f768d4a652969f1b022"
server_key_version: "encryptionkey@0"
I20260925 10:54:51.160492 5014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "23b163d185db4b66b3ab34fe5314518d"
format_stamp: "Formatted at 2026-09-25 10:54:51 on dist-test-slave-nx7s"
server_key: "f679a3279e38d308fe084f44cc9cdb7e"
server_key_iv: "bfe474b5910a4f768d4a652969f1b022"
server_key_version: "encryptionkey@0"
I20260925 10:54:51.165738 5014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260925 10:54:51.169500 5028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:51.171169 5014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:54:51.171387 5014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "23b163d185db4b66b3ab34fe5314518d"
format_stamp: "Formatted at 2026-09-25 10:54:51 on dist-test-slave-nx7s"
server_key: "f679a3279e38d308fe084f44cc9cdb7e"
server_key_iv: "bfe474b5910a4f768d4a652969f1b022"
server_key_version: "encryptionkey@0"
I20260925 10:54:51.171619 5014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:51.204800 5014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:51.205833 5014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:51.206130 5014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:51.224494 5014 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:34863
I20260925 10:54:51.224500 5079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:34863 every 8 connection(s)
I20260925 10:54:51.225956 5014 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:51.230027 5080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:54:51.233121 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 5014
I20260925 10:54:51.233366 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:51.234189 31994 external_mini_cluster.cc:1428] Setting key dc53890db412f922d422656ee6b6f154
I20260925 10:54:51.243227 5080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d: Bootstrap starting.
I20260925 10:54:51.247179 5080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:51.248346 5080 log.cc:824] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:51.249270 5083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:51.236350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55502 (local address 127.31.62.190:34863)
0925 10:54:51.237448 (+ 1098us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:51.237460 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:51.237508 (+ 48us) server_negotiation.cc:407] Connection header received
0925 10:54:51.239219 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:51.239245 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:51.239848 (+ 603us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:51.240238 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:51.240949 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:51.242398 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:51.244236 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:51.245072 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:51.245250 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:51.246528 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:51.246536 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:51.246553 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:54:51.247129 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:51.247226 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:54:51.247248 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:51.247345 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:51.247620 (+ 275us) server_negotiation.cc:299] Negotiation successful
0925 10:54:51.247942 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":415,"threads_started":1}
I20260925 10:54:51.251814 5080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d: No bootstrap required, opened a new log
I20260925 10:54:51.253738 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:54:51.260708 5080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b163d185db4b66b3ab34fe5314518d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34863 } }
I20260925 10:54:51.261221 5080 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:51.261330 5080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b163d185db4b66b3ab34fe5314518d, State: Initialized, Role: FOLLOWER
I20260925 10:54:51.261965 5080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b163d185db4b66b3ab34fe5314518d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34863 } }
I20260925 10:54:51.262188 5080 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:51.262318 5080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:51.262462 5080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:51.264741 5080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b163d185db4b66b3ab34fe5314518d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34863 } }
I20260925 10:54:51.265517 5080 leader_election.cc:302] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23b163d185db4b66b3ab34fe5314518d; no voters:
I20260925 10:54:51.266219 5080 leader_election.cc:288] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:51.266398 5085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:51.267148 5085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [term 1 LEADER]: Becoming Leader. State: Replica: 23b163d185db4b66b3ab34fe5314518d, State: Running, Role: LEADER
I20260925 10:54:51.267699 5085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b163d185db4b66b3ab34fe5314518d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34863 } }
I20260925 10:54:51.268286 5080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:51.270632 5086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23b163d185db4b66b3ab34fe5314518d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b163d185db4b66b3ab34fe5314518d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34863 } } }
I20260925 10:54:51.270602 5087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23b163d185db4b66b3ab34fe5314518d. Latest consensus state: current_term: 1 leader_uuid: "23b163d185db4b66b3ab34fe5314518d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b163d185db4b66b3ab34fe5314518d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34863 } } }
I20260925 10:54:51.271077 5086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:51.271138 5087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:51.273346 5091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:51.279186 5091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:51.288969 5091 catalog_manager.cc:1347] Generated new cluster ID: 9e4be0427f964f5d9d9f6cc0ec53de17
I20260925 10:54:51.289106 5091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:51.308892 5091 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:51.310784 5091 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:51.323426 5091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23b163d185db4b66b3ab34fe5314518d: Generated new TSK 0
I20260925 10:54:51.324348 5091 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:54:51.791937 5083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:51.344806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55516 (local address 127.31.62.190:34863)
0925 10:54:51.345056 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:51.345062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:51.345454 (+ 392us) server_negotiation.cc:407] Connection header received
0925 10:54:51.345716 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:51.345720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:51.345804 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:51.345945 (+ 141us) server_negotiation.cc:226] Negotiated authn=JWT
0925 10:54:51.346519 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:51.347257 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:51.349462 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:51.349921 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:51.350030 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:51.350637 (+ 607us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 10:54:51.791384 (+440747us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 10:54:51.791719 (+ 335us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55516: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":89}
W20260925 10:54:51.792053 5083 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55516: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260925 10:54:51.792387 5109 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:54:51.344651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34863 (local address 127.0.0.1:55516)
0925 10:54:51.345332 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:51.345353 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:51.345529 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:51.345986 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:51.345991 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:51.346004 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0925 10:54:51.346340 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:51.346354 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:51.347430 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:51.347436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:51.349272 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:51.349292 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:51.350125 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:51.350132 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:51.350323 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:51.350375 (+ 52us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 10:54:51.791715 (+441340us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 10:54:51.791976 (+ 261us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.62.190:34863: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":371,"threads_started":1}
I20260925 10:54:51.793452 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 5014
I20260925 10:54:51.795042 5075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:51.898770 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 5014
2026-09-25T10:54:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1174 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (435 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (402 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:54:52 dist-test-slave-nx7s krb5kdc[5124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:54:52 dist-test-slave-nx7s krb5kdc[5124](info): set up 2 sockets
Sep 25 10:54:52 dist-test-slave-nx7s krb5kdc[5124](info): commencing operation
krb5kdc: starting...
W20260925 10:54:53.742009 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.972s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:54:53 dist-test-slave-nx7s krb5kdc[5124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:54:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:54:53.945061 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:42847
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:42847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:54.271502 5137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:54.271840 5137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:54.271970 5137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:54.279892 5137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:54.280028 5137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:54.280077 5137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:54.280126 5137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:54.280169 5137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:54:54.290632 5137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:42847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:42847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:54.291993 5137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:54.293411 5137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:54.305092 5142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:54.305058 5143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:54.305182 5145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:54.306337 5137 server_base.cc:1034] running on GCE node
I20260925 10:54:54.307156 5137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:54.308789 5137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:54.310034 5137 hybrid_clock.cc:648] HybridClock initialized: now 1790333694309968 us; error 53 us; skew 500 ppm
I20260925 10:54:54.310381 5137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:54 dist-test-slave-nx7s krb5kdc[5124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333694, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:54.313565 5137 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:54:54.315019 5137 webserver.cc:460] Webserver started at http://127.31.62.190:45603/ using document root <none> and password file <none>
I20260925 10:54:54.315729 5137 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:54.315850 5137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:54.316140 5137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:54.319108 5137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "3273c1b3ba4544e58543cb9e15ce280f"
format_stamp: "Formatted at 2026-09-25 10:54:54 on dist-test-slave-nx7s"
server_key: "b284d467f2d46de502962051ab4d126b"
server_key_iv: "4ab56b94f6ea5001b7e8ccac76921209"
server_key_version: "encryptionkey@0"
I20260925 10:54:54.319868 5137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "3273c1b3ba4544e58543cb9e15ce280f"
format_stamp: "Formatted at 2026-09-25 10:54:54 on dist-test-slave-nx7s"
server_key: "b284d467f2d46de502962051ab4d126b"
server_key_iv: "4ab56b94f6ea5001b7e8ccac76921209"
server_key_version: "encryptionkey@0"
I20260925 10:54:54.325169 5137 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 10:54:54.328838 5152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:54.330511 5137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:54:54.330700 5137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "3273c1b3ba4544e58543cb9e15ce280f"
format_stamp: "Formatted at 2026-09-25 10:54:54 on dist-test-slave-nx7s"
server_key: "b284d467f2d46de502962051ab4d126b"
server_key_iv: "4ab56b94f6ea5001b7e8ccac76921209"
server_key_version: "encryptionkey@0"
I20260925 10:54:54.330901 5137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:54.347889 5137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:54.352483 5137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:54.352806 5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:54.372180 5137 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:42847
I20260925 10:54:54.372179 5203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:42847 every 8 connection(s)
I20260925 10:54:54.373760 5137 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:54.374904 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 5137
I20260925 10:54:54.375134 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:54.375900 31994 external_mini_cluster.cc:1428] Setting key 98aefe4dd8fe47cf28bc0a7b81673841
I20260925 10:54:54.378768 5204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 10:54:54 dist-test-slave-nx7s krb5kdc[5124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:54.391911 5204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f: Bootstrap starting.
I20260925 10:54:54.396193 5204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:54.397487 5204 log.cc:824] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:54.400786 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:54.378124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50428 (local address 127.31.62.190:42847)
0925 10:54:54.378947 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:54.378963 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:54.379008 (+ 45us) server_negotiation.cc:407] Connection header received
0925 10:54:54.380651 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:54.380676 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:54.381178 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:54.381551 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:54.383312 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:54.384762 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:54.386526 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:54.387289 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:54.387430 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:54.392218 (+ 4788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:54.392230 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:54.392252 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:54.392304 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:54.396123 (+ 3819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:54.396939 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:54.396948 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:54.396957 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:54.397165 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:54.397513 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:54.397518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:54.397521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:54.398240 (+ 719us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:54.398471 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:54.399001 (+ 530us) server_negotiation.cc:299] Negotiation successful
0925 10:54:54.399434 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":355,"threads_started":1}
I20260925 10:54:54.400956 5204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f: No bootstrap required, opened a new log
I20260925 10:54:54.408994 5204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3273c1b3ba4544e58543cb9e15ce280f" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42847 } }
I20260925 10:54:54.409430 5204 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:54.409541 5204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3273c1b3ba4544e58543cb9e15ce280f, State: Initialized, Role: FOLLOWER
I20260925 10:54:54.410207 5204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3273c1b3ba4544e58543cb9e15ce280f" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42847 } }
I20260925 10:54:54.410439 5204 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:54.410581 5204 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:54.410753 5204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:54.412968 5204 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3273c1b3ba4544e58543cb9e15ce280f" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42847 } }
I20260925 10:54:54.413506 5204 leader_election.cc:302] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3273c1b3ba4544e58543cb9e15ce280f; no voters:
I20260925 10:54:54.414113 5204 leader_election.cc:288] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:54.414270 5209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:54.415773 5209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [term 1 LEADER]: Becoming Leader. State: Replica: 3273c1b3ba4544e58543cb9e15ce280f, State: Running, Role: LEADER
I20260925 10:54:54.416327 5209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3273c1b3ba4544e58543cb9e15ce280f" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42847 } }
I20260925 10:54:54.416545 5204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:54.419163 5210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3273c1b3ba4544e58543cb9e15ce280f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3273c1b3ba4544e58543cb9e15ce280f" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42847 } } }
I20260925 10:54:54.419219 5211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3273c1b3ba4544e58543cb9e15ce280f. Latest consensus state: current_term: 1 leader_uuid: "3273c1b3ba4544e58543cb9e15ce280f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3273c1b3ba4544e58543cb9e15ce280f" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42847 } } }
I20260925 10:54:54.419620 5211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:54.419617 5210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:54.423003 5217 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:54.428365 5217 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:54.438504 5217 catalog_manager.cc:1347] Generated new cluster ID: bf42cb9cd6d44a03ba856418ca414109
I20260925 10:54:54.438635 5217 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:54.452776 5217 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:54.454329 5217 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:54.466719 5217 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3273c1b3ba4544e58543cb9e15ce280f: Generated new TSK 0
I20260925 10:54:54.467840 5217 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:54:54.532897 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42847
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:54.877698 5230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:54.878003 5230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:54.878129 5230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:54.885661 5230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:54.885790 5230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:54.885939 5230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:54:54.896152 5230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:54.897498 5230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:54.898803 5230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:54.912617 5238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:54.912648 5236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:54.912532 5235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:54.913913 5230 server_base.cc:1034] running on GCE node
I20260925 10:54:54.914703 5230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:54.915701 5230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:54.916967 5230 hybrid_clock.cc:648] HybridClock initialized: now 1790333694916912 us; error 47 us; skew 500 ppm
I20260925 10:54:54.917389 5230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:54 dist-test-slave-nx7s krb5kdc[5124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333694, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:54.920557 5230 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:54:54.922098 5230 webserver.cc:460] Webserver started at http://127.31.62.129:35151/ using document root <none> and password file <none>
I20260925 10:54:54.922763 5230 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:54.922863 5230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:54.923130 5230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:54.926103 5230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "b67999574c1e4828ba3b1a008bd6ce46"
format_stamp: "Formatted at 2026-09-25 10:54:54 on dist-test-slave-nx7s"
server_key: "5ea8b4cefc655222434b806769fb7a86"
server_key_iv: "517528aa2c53360c3416ccdc2f7be46b"
server_key_version: "encryptionkey@0"
I20260925 10:54:54.926852 5230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "b67999574c1e4828ba3b1a008bd6ce46"
format_stamp: "Formatted at 2026-09-25 10:54:54 on dist-test-slave-nx7s"
server_key: "5ea8b4cefc655222434b806769fb7a86"
server_key_iv: "517528aa2c53360c3416ccdc2f7be46b"
server_key_version: "encryptionkey@0"
I20260925 10:54:54.932200 5230 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260925 10:54:54.936112 5245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:54.937933 5230 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260925 10:54:54.938127 5230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "b67999574c1e4828ba3b1a008bd6ce46"
format_stamp: "Formatted at 2026-09-25 10:54:54 on dist-test-slave-nx7s"
server_key: "5ea8b4cefc655222434b806769fb7a86"
server_key_iv: "517528aa2c53360c3416ccdc2f7be46b"
server_key_version: "encryptionkey@0"
I20260925 10:54:54.938391 5230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:54.959666 5230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:54.964313 5230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:54.964589 5230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:54.965736 5230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:54.967115 5230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:54.967190 5230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:54.967309 5230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:54.967363 5230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:54.997498 5230 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:36083
I20260925 10:54:54.997562 5357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:36083 every 8 connection(s)
I20260925 10:54:54.998816 5230 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:54:55.006067 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 5230
I20260925 10:54:55.006266 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:54:55.006896 31994 external_mini_cluster.cc:1428] Setting key 74829ee4d64f78086961aa4d43d150ac
Sep 25 10:54:55 dist-test-slave-nx7s krb5kdc[5124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333694, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:55.021000 5360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:55.001336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:52877 (local address 127.31.62.190:42847)
0925 10:54:55.001747 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:55.001752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:55.002875 (+ 1123us) server_negotiation.cc:407] Connection header received
0925 10:54:55.004688 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:55.004693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:55.004806 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:55.004924 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:55.006694 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:55.007475 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:55.009718 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:55.010126 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:55.010314 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:55.014471 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:55.014478 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:55.014482 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:55.014526 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:55.017168 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:55.018236 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:55.018242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:55.018244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:55.018381 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:55.018896 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:55.018900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:55.018903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:55.019251 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:55.019386 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:55.020358 (+ 972us) server_negotiation.cc:299] Negotiation successful
0925 10:54:55.020605 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":206,"threads_started":1}
I20260925 10:54:55.022224 5361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:55.001983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:42847 (local address 127.31.62.129:52877)
0925 10:54:55.002697 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:55.002737 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:55.004391 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:55.005202 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:55.005212 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:55.005786 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:55.006473 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:55.006522 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:55.007688 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:55.007694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:55.009511 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:55.009529 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:55.010447 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:55.010455 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:55.010789 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:55.011550 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:55.011587 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:55.014175 (+ 2588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:55.017360 (+ 3185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:55.017372 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:55.017393 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:55.018063 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:55.018501 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:55.018505 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:55.018508 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:55.018758 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:55.019380 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:55.019389 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:55.019879 (+ 490us) client_negotiation.cc:769] Sending connection context
0925 10:54:55.020200 (+ 321us) client_negotiation.cc:240] Negotiation successful
0925 10:54:55.020788 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":278,"threads_started":1}
I20260925 10:54:55.024056 5358 heartbeater.cc:347] Connected to a master server at 127.31.62.190:42847
I20260925 10:54:55.024361 5358 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:55.025182 5358 heartbeater.cc:511] Master 127.31.62.190:42847 requested a full tablet report, sending...
I20260925 10:54:55.027422 5169 ts_manager.cc:194] Registered new tserver with Master: b67999574c1e4828ba3b1a008bd6ce46 (127.31.62.129:36083)
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
I20260925 10:54:55.028965 5169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:52877
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:54:55.054154 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42847
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:55.381878 5364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:55.382159 5364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:55.382289 5364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:55.389750 5364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:55.389876 5364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:55.390033 5364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:54:55.400210 5364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:55.401566 5364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:55.402915 5364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:55.416600 5369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:55.416538 5372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:55.416507 5370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:55.417158 5364 server_base.cc:1034] running on GCE node
I20260925 10:54:55.417953 5364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:55.418952 5364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:55.420156 5364 hybrid_clock.cc:648] HybridClock initialized: now 1790333695420107 us; error 52 us; skew 500 ppm
I20260925 10:54:55.420490 5364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:55 dist-test-slave-nx7s krb5kdc[5124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333695, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:55.423657 5364 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:54:55.425323 5364 webserver.cc:460] Webserver started at http://127.31.62.130:39157/ using document root <none> and password file <none>
I20260925 10:54:55.426046 5364 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:55.426165 5364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:55.426530 5364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:55.429662 5364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "c1ce6c8be2ed42da8f53d6e46cc6e762"
format_stamp: "Formatted at 2026-09-25 10:54:55 on dist-test-slave-nx7s"
server_key: "e0c9a82507d8f434f1ee9541ec1992de"
server_key_iv: "1cbb8d6bb349d18efc688d3877a411ea"
server_key_version: "encryptionkey@0"
I20260925 10:54:55.430508 5364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "c1ce6c8be2ed42da8f53d6e46cc6e762"
format_stamp: "Formatted at 2026-09-25 10:54:55 on dist-test-slave-nx7s"
server_key: "e0c9a82507d8f434f1ee9541ec1992de"
server_key_iv: "1cbb8d6bb349d18efc688d3877a411ea"
server_key_version: "encryptionkey@0"
I20260925 10:54:55.436018 5364 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 10:54:55.439720 5379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:55.441234 5364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:54:55.441392 5364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "c1ce6c8be2ed42da8f53d6e46cc6e762"
format_stamp: "Formatted at 2026-09-25 10:54:55 on dist-test-slave-nx7s"
server_key: "e0c9a82507d8f434f1ee9541ec1992de"
server_key_iv: "1cbb8d6bb349d18efc688d3877a411ea"
server_key_version: "encryptionkey@0"
I20260925 10:54:55.441591 5364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:55.460494 5364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:55.465026 5364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:55.465312 5364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:55.466372 5364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:55.467892 5364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:55.467974 5364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:55.468093 5364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:55.468158 5364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:55.497951 5364 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:45797
I20260925 10:54:55.497992 5491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:45797 every 8 connection(s)
I20260925 10:54:55.499290 5364 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:54:55.504052 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 5364
I20260925 10:54:55.504294 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:54:55.504693 31994 external_mini_cluster.cc:1428] Setting key cae3820f2df2de1edbc4bf6bc633b8f4
Sep 25 10:54:55 dist-test-slave-nx7s krb5kdc[5124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333695, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:55.520452 5360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:55.501935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:42171 (local address 127.31.62.190:42847)
0925 10:54:55.502116 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:55.502120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:55.503361 (+ 1241us) server_negotiation.cc:407] Connection header received
0925 10:54:55.505265 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:55.505270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:55.505339 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:55.505436 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:55.507096 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:55.508105 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:55.509692 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:55.510262 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:55.510380 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:55.514083 (+ 3703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:55.514088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:55.514091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:55.514127 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:55.516727 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:55.517744 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:55.517748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:55.517751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:55.517886 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:55.518421 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:55.518427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:55.518429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:55.518800 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:55.518933 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:55.519897 (+ 964us) server_negotiation.cc:299] Negotiation successful
0925 10:54:55.520112 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 10:54:55.521955 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:55.502461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:42847 (local address 127.31.62.130:42171)
0925 10:54:55.503189 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:55.503233 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:55.504984 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:55.505676 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:55.505685 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:55.506217 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:55.506913 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:55.506932 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:55.508247 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:55.508250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:55.509560 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:55.509572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:55.510449 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:55.510453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:55.510660 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:55.511299 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:55.511323 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:55.513812 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:55.516927 (+ 3115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:55.516938 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:55.516959 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:55.517579 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:55.517991 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:55.517996 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:55.517999 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:55.518255 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:55.518932 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:55.518941 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:55.519439 (+ 498us) client_negotiation.cc:769] Sending connection context
0925 10:54:55.519749 (+ 310us) client_negotiation.cc:240] Negotiation successful
0925 10:54:55.520366 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":276,"threads_started":1}
I20260925 10:54:55.523720 5492 heartbeater.cc:347] Connected to a master server at 127.31.62.190:42847
I20260925 10:54:55.524094 5492 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
I20260925 10:54:55.524993 5492 heartbeater.cc:511] Master 127.31.62.190:42847 requested a full tablet report, sending...
I20260925 10:54:55.526647 5169 ts_manager.cc:194] Registered new tserver with Master: c1ce6c8be2ed42da8f53d6e46cc6e762 (127.31.62.130:45797)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
I20260925 10:54:55.527729 5169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:42171
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:54:55.553619 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42847
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:55.877913 5497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:55.878221 5497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:55.878314 5497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:55.885744 5497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:55.885870 5497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:55.885980 5497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:54:55.895946 5497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:55.897320 5497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:55.898595 5497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:55.911792 5502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:55.911734 5503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:55.911760 5505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:55.913020 5497 server_base.cc:1034] running on GCE node
I20260925 10:54:55.913667 5497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:55.914645 5497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:55.915861 5497 hybrid_clock.cc:648] HybridClock initialized: now 1790333695915794 us; error 66 us; skew 500 ppm
I20260925 10:54:55.916203 5497 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:55 dist-test-slave-nx7s krb5kdc[5124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333695, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:55.919512 5497 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:54:55.921182 5497 webserver.cc:460] Webserver started at http://127.31.62.131:34763/ using document root <none> and password file <none>
I20260925 10:54:55.921923 5497 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:55.922039 5497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:55.922341 5497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:55.925390 5497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "0cb4173127064ab4a4356e1d861def45"
format_stamp: "Formatted at 2026-09-25 10:54:55 on dist-test-slave-nx7s"
server_key: "e6167ca1a2db6e0c50d6e891b94dfb32"
server_key_iv: "7b0369a1211e49f30c2fe29a6c939b49"
server_key_version: "encryptionkey@0"
I20260925 10:54:55.926170 5497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "0cb4173127064ab4a4356e1d861def45"
format_stamp: "Formatted at 2026-09-25 10:54:55 on dist-test-slave-nx7s"
server_key: "e6167ca1a2db6e0c50d6e891b94dfb32"
server_key_iv: "7b0369a1211e49f30c2fe29a6c939b49"
server_key_version: "encryptionkey@0"
I20260925 10:54:55.931576 5497 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:54:55.935365 5512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:55.936993 5497 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 10:54:55.937178 5497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "0cb4173127064ab4a4356e1d861def45"
format_stamp: "Formatted at 2026-09-25 10:54:55 on dist-test-slave-nx7s"
server_key: "e6167ca1a2db6e0c50d6e891b94dfb32"
server_key_iv: "7b0369a1211e49f30c2fe29a6c939b49"
server_key_version: "encryptionkey@0"
I20260925 10:54:55.937388 5497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:55.956245 5497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:55.960628 5497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:55.960961 5497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:55.961999 5497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:55.963474 5497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:55.963552 5497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:55.963678 5497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:55.963729 5497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:55.993975 5497 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:45781
I20260925 10:54:55.994023 5624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:45781 every 8 connection(s)
I20260925 10:54:55.995292 5497 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:54:56.003396 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 5497
I20260925 10:54:56.003575 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:54:56.004010 31994 external_mini_cluster.cc:1428] Setting key cc3c568b88f144267afcc2bb9367d118
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333695, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:54:56.017434 5360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:55.997958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:50077 (local address 127.31.62.190:42847)
0925 10:54:55.998166 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:55.998172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:55.999438 (+ 1266us) server_negotiation.cc:407] Connection header received
0925 10:54:56.001968 (+ 2530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:56.001973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:56.002045 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:56.002140 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:56.004370 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.005217 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.007014 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.007418 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.007516 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:56.011184 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:56.011189 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:56.011192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:56.011228 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:56.013890 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:56.014856 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:56.014861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:56.014863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:56.015014 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:56.015594 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:56.015599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:56.015601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:56.015880 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:56.015994 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:56.016951 (+ 957us) server_negotiation.cc:299] Negotiation successful
0925 10:54:56.017162 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 10:54:56.018957 5627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:55.998438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:42847 (local address 127.31.62.131:50077)
0925 10:54:55.999250 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:55.999306 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:56.001608 (+ 2302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:56.002557 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:56.002570 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:56.003451 (+ 881us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:56.004169 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:56.004193 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.005385 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.005392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:56.006882 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:56.006895 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.007661 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.007667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:56.007887 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:56.008653 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:56.008677 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:56.010951 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:56.014102 (+ 3151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:56.014111 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:56.014129 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:56.014709 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:56.015253 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:56.015257 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:56.015259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:56.015460 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:56.016010 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:56.016017 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:56.016458 (+ 441us) client_negotiation.cc:769] Sending connection context
0925 10:54:56.016756 (+ 298us) client_negotiation.cc:240] Negotiation successful
0925 10:54:56.017353 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":308,"threads_started":1}
I20260925 10:54:56.020479 5625 heartbeater.cc:347] Connected to a master server at 127.31.62.190:42847
I20260925 10:54:56.020799 5625 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:56.021654 5625 heartbeater.cc:511] Master 127.31.62.190:42847 requested a full tablet report, sending...
I20260925 10:54:56.023257 5169 ts_manager.cc:194] Registered new tserver with Master: 0cb4173127064ab4a4356e1d861def45 (127.31.62.131:45781)
I20260925 10:54:56.024061 5169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:50077
I20260925 10:54:56.032788 5358 heartbeater.cc:503] Master 127.31.62.190:42847 was elected leader, sending a full tablet report...
I20260925 10:54:56.032967 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/test-admin.keytab.
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:56.058929 31994 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260925 10:54:56.064055 5632 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:54:56.061676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59474 (local address 127.31.62.129:36083)
0925 10:54:56.062342 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:56.062349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:56.062373 (+ 24us) server_negotiation.cc:407] Connection header received
0925 10:54:56.062428 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:56.062434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:56.062655 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:56.062816 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:56.063689 (+ 873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59474: BlockingRecv error: recv got EOF from 127.0.0.1:59474 (error 108)
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":305,"threads_started":1}
W20260925 10:54:56.064258 31994 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/krb5cc' of type FILE
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:56.065970 31994 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:56.067371 31994 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:54:56.084044 5632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:56.069387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59480 (local address 127.31.62.129:36083)
0925 10:54:56.069600 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:56.069605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:56.069676 (+ 71us) server_negotiation.cc:407] Connection header received
0925 10:54:56.069891 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:56.069896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:56.069960 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:56.070049 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:56.071027 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.072091 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.073531 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.074185 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.074293 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:56.077971 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:56.077985 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:56.078002 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:56.078041 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:56.080994 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:56.081757 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:56.081764 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:56.081767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:56.081903 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:56.082387 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:56.082391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:56.082393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:56.082820 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:56.083038 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:56.083444 (+ 406us) server_negotiation.cc:299] Negotiation successful
0925 10:54:56.083763 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260925 10:54:56.090219 5632 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:54:56.088456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59484 (local address 127.31.62.129:36083)
0925 10:54:56.088650 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:56.088657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:56.088909 (+ 252us) server_negotiation.cc:407] Connection header received
0925 10:54:56.089076 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:56.089079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:56.089139 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:56.089226 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:56.089999 (+ 773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59484: BlockingRecv error: recv got EOF from 127.0.0.1:59484 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260925 10:54:56.090621 31994 init.cc:244] Error while opening credential cache '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/krb5cc)
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:56.092162 31994 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:56.093613 31994 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:54:56.108891 5632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:56.095402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59500 (local address 127.31.62.129:36083)
0925 10:54:56.095650 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:56.095655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:56.095754 (+ 99us) server_negotiation.cc:407] Connection header received
0925 10:54:56.095888 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:56.095892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:56.095955 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:56.096039 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:56.097050 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.097784 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.099018 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.099503 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.099640 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:56.103381 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:56.103386 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:56.103395 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:56.103448 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:56.106363 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:56.107123 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:56.107127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:56.107129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:56.107263 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:56.107617 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:56.107622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:56.107626 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:56.107886 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:56.108085 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:56.108393 (+ 308us) server_negotiation.cc:299] Negotiation successful
0925 10:54:56.108605 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260925 10:54:56.113395 5632 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:54:56.111692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59510 (local address 127.31.62.129:36083)
0925 10:54:56.111881 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:56.111886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:56.112083 (+ 197us) server_negotiation.cc:407] Connection header received
0925 10:54:56.112230 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:56.112234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:56.112318 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:56.112407 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:56.113158 (+ 751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59510: BlockingRecv error: recv got EOF from 127.0.0.1:59510 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260925 10:54:56.113528 31994 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790333647382755-31994-0/krb5kdc/krb5cc' of type FILE
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:56.115080 31994 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:56.116494 31994 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:54:56.132302 5632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:56.117916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59524 (local address 127.31.62.129:36083)
0925 10:54:56.118121 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:56.118127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:56.118411 (+ 284us) server_negotiation.cc:407] Connection header received
0925 10:54:56.118539 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:56.118542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:56.118598 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:56.118676 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:56.119569 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.120330 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.121842 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.122297 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.122390 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:56.126143 (+ 3753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:56.126148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:56.126151 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:56.126187 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:56.129342 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:56.130154 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:56.130158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:56.130161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:56.130351 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:56.130774 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:56.130778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:56.130780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:56.131224 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:56.131445 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:56.131739 (+ 294us) server_negotiation.cc:299] Negotiation successful
0925 10:54:56.131910 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 10:54:56.133910 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 5230
I20260925 10:54:56.146942 5632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:56.134567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59534 (local address 127.31.62.129:36083)
0925 10:54:56.134740 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:56.134746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:56.135078 (+ 332us) server_negotiation.cc:407] Connection header received
0925 10:54:56.135268 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:56.135272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:56.135332 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:56.135402 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:56.136477 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.137293 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.138554 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.138994 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.139156 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:56.141388 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:56.141393 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:56.141404 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:56.141445 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:56.144304 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:56.145048 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:56.145053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:56.145057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:56.145220 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:56.145590 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:56.145594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:56.145596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:56.145890 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:56.146081 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:56.146453 (+ 372us) server_negotiation.cc:299] Negotiation successful
0925 10:54:56.146624 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 10:54:56.147462 5353 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:56.234758 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 5230
I20260925 10:54:56.248567 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 5364
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:54:56.264979 5645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:56.249693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.31.62.130:45797)
0925 10:54:56.250202 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:56.250211 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:56.250239 (+ 28us) server_negotiation.cc:407] Connection header received
0925 10:54:56.250298 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:56.250304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:56.250518 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:56.250712 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:56.251815 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.252836 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.254274 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.254861 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.254978 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:56.258807 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:56.258821 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:56.258837 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:56.258874 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:56.261857 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:56.262601 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:56.262609 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:56.262611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:56.262748 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:56.263313 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:56.263317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:56.263319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:56.263697 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:56.263989 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:56.264389 (+ 400us) server_negotiation.cc:299] Negotiation successful
0925 10:54:56.264616 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":245,"threads_started":1}
I20260925 10:54:56.266299 5487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:56.349764 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 5364
I20260925 10:54:56.362483 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 5497
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:54:56.379549 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:56.363437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48598 (local address 127.31.62.131:45781)
0925 10:54:56.364076 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:56.364083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:56.364112 (+ 29us) server_negotiation.cc:407] Connection header received
0925 10:54:56.364184 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:56.364192 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:56.364468 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:56.364612 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:56.365818 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.366821 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.368218 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:56.368835 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:56.368981 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:56.373391 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:56.373405 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:56.373421 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:56.373458 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:56.376466 (+ 3008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:56.377396 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:56.377404 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:56.377406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:56.377558 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:56.377958 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:56.377961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:56.377963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:56.378392 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:56.378632 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:56.378929 (+ 297us) server_negotiation.cc:299] Negotiation successful
0925 10:54:56.379220 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":317,"threads_started":1}
I20260925 10:54:56.380891 5620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:56.463651 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 5497
I20260925 10:54:56.476465 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 5137
I20260925 10:54:56.477648 5199 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:54:56.580034 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 5137
2026-09-25T10:54:56Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3846 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5652](info): set up 2 sockets
Sep 25 10:54:56 dist-test-slave-nx7s krb5kdc[5652](info): commencing operation
krb5kdc: starting...
W20260925 10:54:57.593722 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.977s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:54:57 dist-test-slave-nx7s krb5kdc[5652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:54:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:54:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.190.keytab.
Entry for principal oryx/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.190.keytab.
I20260925 10:54:57.796826 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:40377
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.190.keytab
--principal=oryx/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:33393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:40377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:58.117481 5665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:58.117802 5665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:58.117914 5665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:58.125231 5665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:54:58.125339 5665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:58.125391 5665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:58.125475 5665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:54:58.125536 5665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:54:58.135501 5665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:33393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:40377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.190.keytab
--principal=oryx/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:40377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:58.136881 5665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:58.138281 5665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:58.150007 5670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:58.149974 5671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:58.150041 5673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:58.151264 5665 server_base.cc:1034] running on GCE node
I20260925 10:54:58.152056 5665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:58.153636 5665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:58.154866 5665 hybrid_clock.cc:648] HybridClock initialized: now 1790333698154814 us; error 37 us; skew 500 ppm
I20260925 10:54:58.155268 5665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:58 dist-test-slave-nx7s krb5kdc[5652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333698, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:58.158305 5665 init.cc:377] Logged in from keytab as oryx/127.31.62.190@KRBTEST.COM (short username oryx)
I20260925 10:54:58.159823 5665 webserver.cc:460] Webserver started at http://127.31.62.190:44349/ using document root <none> and password file <none>
I20260925 10:54:58.160516 5665 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:58.160624 5665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:58.160936 5665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:58.163985 5665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "44235815436447c3ab94d41adcbe92c5"
format_stamp: "Formatted at 2026-09-25 10:54:58 on dist-test-slave-nx7s"
server_key: "8b6c7eabdeeede0b3b004b454ec03336"
server_key_iv: "59bd0fb6c4b440d764004d49437e0115"
server_key_version: "encryptionkey@0"
I20260925 10:54:58.164744 5665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "44235815436447c3ab94d41adcbe92c5"
format_stamp: "Formatted at 2026-09-25 10:54:58 on dist-test-slave-nx7s"
server_key: "8b6c7eabdeeede0b3b004b454ec03336"
server_key_iv: "59bd0fb6c4b440d764004d49437e0115"
server_key_version: "encryptionkey@0"
I20260925 10:54:58.170348 5665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260925 10:54:58.174636 5680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:58.176499 5665 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 10:54:58.176708 5665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "44235815436447c3ab94d41adcbe92c5"
format_stamp: "Formatted at 2026-09-25 10:54:58 on dist-test-slave-nx7s"
server_key: "8b6c7eabdeeede0b3b004b454ec03336"
server_key_iv: "59bd0fb6c4b440d764004d49437e0115"
server_key_version: "encryptionkey@0"
I20260925 10:54:58.176949 5665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:58.199532 5665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:58.203980 5665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:58.204263 5665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:58.223549 5665 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:40377
I20260925 10:54:58.223577 5731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:40377 every 8 connection(s)
I20260925 10:54:58.225016 5665 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:54:58.226533 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 5665
I20260925 10:54:58.226781 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:54:58.227558 31994 external_mini_cluster.cc:1428] Setting key a1465481f4c4f421112a616f64ea191c
I20260925 10:54:58.230031 5732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 10:54:58 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
I20260925 10:54:58.244175 5732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5: Bootstrap starting.
I20260925 10:54:58.248831 5732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5: Neither blocks nor log segments found. Creating new log.
I20260925 10:54:58.250312 5732 log.cc:824] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5: Log is configured to *not* fsync() on all Append() calls
I20260925 10:54:58.253795 5732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5: No bootstrap required, opened a new log
I20260925 10:54:58.254679 5735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:58.229825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35262 (local address 127.31.62.190:40377)
0925 10:54:58.230623 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:58.230639 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:58.230689 (+ 50us) server_negotiation.cc:407] Connection header received
0925 10:54:58.232837 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:58.232891 (+ 54us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:58.233492 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:58.233980 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:58.235572 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:58.237714 (+ 2142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:58.239260 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:58.240381 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:58.240606 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:58.245222 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:58.245234 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:58.245257 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:58.245315 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:58.249535 (+ 4220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:58.250375 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:58.250386 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:58.250396 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:58.250641 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:58.251147 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:58.251158 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:58.251161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:58.251841 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:58.252172 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:58.252762 (+ 590us) server_negotiation.cc:299] Negotiation successful
0925 10:54:58.253428 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":338,"threads_started":1}
I20260925 10:54:58.262257 5732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } }
I20260925 10:54:58.262786 5732 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:54:58.262887 5732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44235815436447c3ab94d41adcbe92c5, State: Initialized, Role: FOLLOWER
I20260925 10:54:58.263514 5732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } }
I20260925 10:54:58.263749 5732 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:54:58.263883 5732 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:54:58.264041 5732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:54:58.266614 5732 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } }
I20260925 10:54:58.267232 5732 leader_election.cc:302] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44235815436447c3ab94d41adcbe92c5; no voters:
I20260925 10:54:58.267851 5732 leader_election.cc:288] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:54:58.268002 5737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:54:58.268672 5737 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [term 1 LEADER]: Becoming Leader. State: Replica: 44235815436447c3ab94d41adcbe92c5, State: Running, Role: LEADER
I20260925 10:54:58.269261 5737 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } }
I20260925 10:54:58.269948 5732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:54:58.272063 5738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } }
I20260925 10:54:58.272037 5739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44235815436447c3ab94d41adcbe92c5. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } }
I20260925 10:54:58.272439 5739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:58.272439 5738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:54:58.273468 5743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:54:58.279466 5743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:54:58.290856 5743 catalog_manager.cc:1347] Generated new cluster ID: 86416e3acf79441b9253cc5f1538d331
I20260925 10:54:58.291002 5743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:54:58.312973 5743 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:54:58.315001 5743 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:54:58.330682 5743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5: Generated new TSK 0
I20260925 10:54:58.331866 5743 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.129.keytab.
Entry for principal oryx/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.129.keytab.
I20260925 10:54:58.381012 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:40377
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.129.keytab
--principal=oryx/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:33393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:58.711817 5758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:58.712116 5758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:58.712210 5758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:58.719445 5758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:58.719560 5758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:58.719681 5758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:54:58.729360 5758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:33393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.129.keytab
--principal=oryx/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:40377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:58.730582 5758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:58.731798 5758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:58.744889 5766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:58.744892 5763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:58.744892 5764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:58.750748 5758 server_base.cc:1034] running on GCE node
I20260925 10:54:58.751467 5758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:58.752565 5758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:58.753823 5758 hybrid_clock.cc:648] HybridClock initialized: now 1790333698753738 us; error 69 us; skew 500 ppm
I20260925 10:54:58.754284 5758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:58 dist-test-slave-nx7s krb5kdc[5652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333698, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:58.757583 5758 init.cc:377] Logged in from keytab as oryx/127.31.62.129@KRBTEST.COM (short username oryx)
I20260925 10:54:58.759212 5758 webserver.cc:460] Webserver started at http://127.31.62.129:40919/ using document root <none> and password file <none>
I20260925 10:54:58.759963 5758 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:58.760089 5758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:58.760382 5758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:58.763509 5758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11"
format_stamp: "Formatted at 2026-09-25 10:54:58 on dist-test-slave-nx7s"
server_key: "626041e35044662ccd598c37e015879a"
server_key_iv: "878f9a60c4e53c7dc035b98a07a4a9b0"
server_key_version: "encryptionkey@0"
I20260925 10:54:58.764298 5758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11"
format_stamp: "Formatted at 2026-09-25 10:54:58 on dist-test-slave-nx7s"
server_key: "626041e35044662ccd598c37e015879a"
server_key_iv: "878f9a60c4e53c7dc035b98a07a4a9b0"
server_key_version: "encryptionkey@0"
I20260925 10:54:58.769948 5758 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 10:54:58.774083 5773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:58.775954 5758 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260925 10:54:58.776160 5758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11"
format_stamp: "Formatted at 2026-09-25 10:54:58 on dist-test-slave-nx7s"
server_key: "626041e35044662ccd598c37e015879a"
server_key_iv: "878f9a60c4e53c7dc035b98a07a4a9b0"
server_key_version: "encryptionkey@0"
I20260925 10:54:58.776404 5758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:58.808184 5758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:58.812587 5758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:58.812877 5758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:58.813853 5758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:58.815207 5758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:58.815281 5758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:58.815362 5758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:58.815410 5758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:58.846663 5758 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:34941
I20260925 10:54:58.846751 5885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:34941 every 8 connection(s)
I20260925 10:54:58.847942 5758 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:54:58.851805 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 5758
I20260925 10:54:58.852028 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:54:58.852814 31994 external_mini_cluster.cc:1428] Setting key 484a6bc97a6e4c06e773a61dca3fadb0
Sep 25 10:54:58 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333698, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
I20260925 10:54:58.868290 5888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:58.850749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:55227 (local address 127.31.62.190:40377)
0925 10:54:58.851233 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:58.851238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:58.852318 (+ 1080us) server_negotiation.cc:407] Connection header received
0925 10:54:58.854481 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:58.854486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:58.854558 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:58.854654 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:58.856206 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:58.856973 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:58.858570 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:58.858963 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:58.859080 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:58.862478 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:58.862483 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:58.862487 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:58.862520 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:58.865139 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:58.865947 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:58.865951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:58.865954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:58.866085 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:58.866521 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:58.866525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:58.866527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:58.866805 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:58.866948 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:58.867707 (+ 759us) server_negotiation.cc:299] Negotiation successful
0925 10:54:58.867931 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":213,"threads_started":1}
I20260925 10:54:58.869362 5889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:58.851221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.129:55227)
0925 10:54:58.852041 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:58.852161 (+ 120us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:58.854187 (+ 2026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:58.854925 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:58.854935 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:58.855447 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:58.856022 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:58.856039 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:58.857129 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:58.857133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:58.858413 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:58.858424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:58.859196 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:58.859203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:58.859431 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:58.860067 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:58.860089 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:58.862238 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:58.865309 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:58.865320 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:58.865347 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:58.865815 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:58.866195 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:58.866200 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:58.866204 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:58.866407 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:58.866936 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:58.866945 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:58.867329 (+ 384us) client_negotiation.cc:769] Sending connection context
0925 10:54:58.867582 (+ 253us) client_negotiation.cc:240] Negotiation successful
0925 10:54:58.868176 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":312,"threads_started":1}
WARNING: no policy specified for oryx/127.31.62.130@KRBTEST.COM; defaulting to no policy
I20260925 10:54:58.871050 5886 heartbeater.cc:347] Connected to a master server at 127.31.62.190:40377
I20260925 10:54:58.871414 5886 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:58.872259 5886 heartbeater.cc:511] Master 127.31.62.190:40377 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.62.130@KRBTEST.COM" created.
I20260925 10:54:58.874596 5697 ts_manager.cc:194] Registered new tserver with Master: 1a0685c684ef4ad2a1d95ef3cf1f2a11 (127.31.62.129:34941)
I20260925 10:54:58.876323 5697 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:55227
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.130.keytab.
Entry for principal oryx/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.130.keytab.
I20260925 10:54:58.898689 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:40377
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.130.keytab
--principal=oryx/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:33393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:59.226143 5892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:59.226562 5892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:59.226769 5892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:59.234361 5892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:59.234483 5892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:59.234627 5892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:54:59.245146 5892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:33393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.130.keytab
--principal=oryx/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:40377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:59.246495 5892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:59.247818 5892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:59.261396 5898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:59.261449 5900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:59.261579 5897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:59.262773 5892 server_base.cc:1034] running on GCE node
I20260925 10:54:59.263448 5892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:59.264451 5892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:59.265666 5892 hybrid_clock.cc:648] HybridClock initialized: now 1790333699265595 us; error 64 us; skew 500 ppm
I20260925 10:54:59.266009 5892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:59 dist-test-slave-nx7s krb5kdc[5652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333699, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:59.269176 5892 init.cc:377] Logged in from keytab as oryx/127.31.62.130@KRBTEST.COM (short username oryx)
I20260925 10:54:59.270735 5892 webserver.cc:460] Webserver started at http://127.31.62.130:34745/ using document root <none> and password file <none>
I20260925 10:54:59.271474 5892 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:59.271593 5892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:59.271886 5892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:59.274902 5892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "5017255eb8f34c7abd7e4ec3f63dc84f"
format_stamp: "Formatted at 2026-09-25 10:54:59 on dist-test-slave-nx7s"
server_key: "aaa7b4d4be8eb8b715be8b32ed503ad2"
server_key_iv: "168fcaa6b00d22daf69877f9688d2ea6"
server_key_version: "encryptionkey@0"
I20260925 10:54:59.275734 5892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "5017255eb8f34c7abd7e4ec3f63dc84f"
format_stamp: "Formatted at 2026-09-25 10:54:59 on dist-test-slave-nx7s"
server_key: "aaa7b4d4be8eb8b715be8b32ed503ad2"
server_key_iv: "168fcaa6b00d22daf69877f9688d2ea6"
server_key_version: "encryptionkey@0"
I20260925 10:54:59.281533 5892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 10:54:59.285619 5907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:59.287334 5892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:54:59.287528 5892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "5017255eb8f34c7abd7e4ec3f63dc84f"
format_stamp: "Formatted at 2026-09-25 10:54:59 on dist-test-slave-nx7s"
server_key: "aaa7b4d4be8eb8b715be8b32ed503ad2"
server_key_iv: "168fcaa6b00d22daf69877f9688d2ea6"
server_key_version: "encryptionkey@0"
I20260925 10:54:59.287760 5892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:59.319420 5892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:59.323782 5892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:59.324074 5892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:59.325126 5892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:59.326506 5892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:59.326584 5892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:59.326694 5892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:59.326745 5892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:59.358793 5892 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:44351
I20260925 10:54:59.358865 6019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:44351 every 8 connection(s)
I20260925 10:54:59.360430 5892 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:54:59.369290 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 5892
I20260925 10:54:59.369481 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:54:59.370038 31994 external_mini_cluster.cc:1428] Setting key 808d9efe94a4929d3f94a118c77a10f8
Sep 25 10:54:59 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333699, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
I20260925 10:54:59.382855 5888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:59.363250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:41117 (local address 127.31.62.190:40377)
0925 10:54:59.363786 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:59.363794 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:59.364816 (+ 1022us) server_negotiation.cc:407] Connection header received
0925 10:54:59.366756 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:59.366760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:59.366833 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:59.366927 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:59.368485 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:59.369284 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:59.371429 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:59.372068 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:59.372201 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:59.376416 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:59.376422 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:59.376427 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:59.376465 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:59.379399 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:59.380428 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:59.380433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:59.380437 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:59.380594 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:59.381076 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:59.381079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:59.381081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:59.381418 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:59.381546 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:59.382428 (+ 882us) server_negotiation.cc:299] Negotiation successful
0925 10:54:59.382601 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384}
I20260925 10:54:59.384274 6022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:59.363802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.130:41117)
0925 10:54:59.364635 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:59.364681 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:59.366436 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:59.367127 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:59.367136 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:59.367658 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:59.368284 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:59.368302 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:59.369464 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:59.369469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:59.371292 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:59.371305 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:59.372349 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:59.372357 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:59.372678 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:59.373473 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:59.373507 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:59.376130 (+ 2623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:59.379606 (+ 3476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:59.379616 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:59.379637 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:59.380274 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:59.380699 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:59.380704 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:59.380708 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:59.380980 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:59.381539 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:59.381547 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:59.382000 (+ 453us) client_negotiation.cc:769] Sending connection context
0925 10:54:59.382319 (+ 319us) client_negotiation.cc:240] Negotiation successful
0925 10:54:59.382869 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":399,"threads_started":1}
I20260925 10:54:59.386016 6020 heartbeater.cc:347] Connected to a master server at 127.31.62.190:40377
I20260925 10:54:59.386515 6020 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:59.387348 6020 heartbeater.cc:511] Master 127.31.62.190:40377 requested a full tablet report, sending...
I20260925 10:54:59.389020 5697 ts_manager.cc:194] Registered new tserver with Master: 5017255eb8f34c7abd7e4ec3f63dc84f (127.31.62.130:44351)
I20260925 10:54:59.389859 5697 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:41117
WARNING: no policy specified for oryx/127.31.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.131.keytab.
Entry for principal oryx/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.131.keytab.
I20260925 10:54:59.422195 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:40377
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.131.keytab
--principal=oryx/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:33393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:54:59.747648 6025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:54:59.747947 6025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:54:59.748082 6025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:54:59.755481 6025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:54:59.755632 6025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:54:59.755796 6025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:54:59.765822 6025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:33393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.131.keytab
--principal=oryx/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:40377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:54:59.767172 6025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:54:59.768481 6025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:54:59.781822 6030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:59.781900 6031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:54:59.782271 6033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:54:59.782842 6025 server_base.cc:1034] running on GCE node
I20260925 10:54:59.783556 6025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:54:59.784579 6025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:54:59.785846 6025 hybrid_clock.cc:648] HybridClock initialized: now 1790333699785779 us; error 69 us; skew 500 ppm
I20260925 10:54:59.786187 6025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:54:59 dist-test-slave-nx7s krb5kdc[5652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333699, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:54:59.789623 6025 init.cc:377] Logged in from keytab as oryx/127.31.62.131@KRBTEST.COM (short username oryx)
I20260925 10:54:59.791153 6025 webserver.cc:460] Webserver started at http://127.31.62.131:39667/ using document root <none> and password file <none>
I20260925 10:54:59.791891 6025 fs_manager.cc:359] Metadata directory not provided
I20260925 10:54:59.792008 6025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:54:59.792285 6025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:54:59.795243 6025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "f727914630b045db8981c6190cd467dc"
format_stamp: "Formatted at 2026-09-25 10:54:59 on dist-test-slave-nx7s"
server_key: "20132df213316c986ce89e3bd953dd99"
server_key_iv: "c0daec8663ed5ad83c8433835c19c3cd"
server_key_version: "encryptionkey@0"
I20260925 10:54:59.796029 6025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "f727914630b045db8981c6190cd467dc"
format_stamp: "Formatted at 2026-09-25 10:54:59 on dist-test-slave-nx7s"
server_key: "20132df213316c986ce89e3bd953dd99"
server_key_iv: "c0daec8663ed5ad83c8433835c19c3cd"
server_key_version: "encryptionkey@0"
I20260925 10:54:59.801355 6025 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260925 10:54:59.804744 6040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:59.806150 6025 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:54:59.806298 6025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "f727914630b045db8981c6190cd467dc"
format_stamp: "Formatted at 2026-09-25 10:54:59 on dist-test-slave-nx7s"
server_key: "20132df213316c986ce89e3bd953dd99"
server_key_iv: "c0daec8663ed5ad83c8433835c19c3cd"
server_key_version: "encryptionkey@0"
I20260925 10:54:59.806473 6025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:54:59.824424 6025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:54:59.828464 6025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:54:59.828723 6025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:54:59.829772 6025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:54:59.831159 6025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:54:59.831244 6025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:59.831365 6025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:54:59.831416 6025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:54:59.866570 6025 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:34571
I20260925 10:54:59.866608 6152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:34571 every 8 connection(s)
I20260925 10:54:59.867797 6025 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:54:59.872916 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 6025
I20260925 10:54:59.873097 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:54:59.873489 31994 external_mini_cluster.cc:1428] Setting key 0a3907d8391b46b246c2b411f379f7b3
I20260925 10:54:59.879490 5886 heartbeater.cc:503] Master 127.31.62.190:40377 was elected leader, sending a full tablet report...
Sep 25 10:54:59 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333699, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
I20260925 10:54:59.888083 5888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:59.870347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:32873 (local address 127.31.62.190:40377)
0925 10:54:59.870545 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 10:54:59.870549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:54:59.871900 (+ 1351us) server_negotiation.cc:407] Connection header received
0925 10:54:59.873749 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:54:59.873755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:54:59.873847 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:54:59.873976 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:54:59.875758 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:59.876685 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:59.878248 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:54:59.878654 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:54:59.878764 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:59.882500 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:54:59.882504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:54:59.882508 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:54:59.882540 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:54:59.885047 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:59.885772 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:59.885776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:59.885779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:59.885900 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:54:59.886308 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:54:59.886311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:54:59.886313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:54:59.886583 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:54:59.886710 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 10:54:59.887534 (+ 824us) server_negotiation.cc:299] Negotiation successful
0925 10:54:59.887759 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 10:54:59.889385 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:54:59.870977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.131:32873)
0925 10:54:59.871742 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0925 10:54:59.871786 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 10:54:59.873460 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:54:59.874236 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:54:59.874246 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:54:59.874872 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:54:59.875582 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:59.875608 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:59.876831 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:59.876835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:59.878103 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:54:59.878115 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:54:59.878878 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:54:59.878885 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:54:59.879118 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:54:59.879833 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:54:59.879887 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:54:59.882288 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:54:59.885187 (+ 2899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:59.885198 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:59.885216 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:59.885642 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:59.886001 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:54:59.886006 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:54:59.886010 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:54:59.886198 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:54:59.886710 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:54:59.886719 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:54:59.887068 (+ 349us) client_negotiation.cc:769] Sending connection context
0925 10:54:59.887376 (+ 308us) client_negotiation.cc:240] Negotiation successful
0925 10:54:59.887953 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":321,"threads_started":1}
I20260925 10:54:59.890789 6153 heartbeater.cc:347] Connected to a master server at 127.31.62.190:40377
I20260925 10:54:59.891106 6153 heartbeater.cc:464] Registering TS with master...
I20260925 10:54:59.891911 6153 heartbeater.cc:511] Master 127.31.62.190:40377 requested a full tablet report, sending...
I20260925 10:54:59.893406 5697 ts_manager.cc:194] Registered new tserver with Master: f727914630b045db8981c6190cd467dc (127.31.62.131:34571)
I20260925 10:54:59.894210 5697 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:32873
I20260925 10:54:59.901820 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.62.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.62.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.62.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.189.keytab.
Entry for principal oryx/127.31.62.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.189.keytab.
I20260925 10:54:59.949244 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.189:36481
--webserver_interface=127.31.62.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.189.keytab
--principal=oryx/127.31.62.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:33393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:40377,127.31.62.189:36481
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:55:00.281394 6158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:00.281765 6158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:00.281911 6158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:00.289705 6158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:55:00.289837 6158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:00.289896 6158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:00.289956 6158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:55:00.290006 6158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:55:00.300206 6158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:33393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:40377,127.31.62.189:36481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.189.keytab
--principal=oryx/127.31.62.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.189:36481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.62.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
W20260925 10:55:00.300557 6158 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.31.62.190:40377,127.31.62.189:36481'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260925 10:55:00.301832 6158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:00.303170 6158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:00.315129 6164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:00.315068 6166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:00.315085 6163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:00.315824 6158 server_base.cc:1034] running on GCE node
I20260925 10:55:00.316581 6158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:00.318005 6158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:00.319229 6158 hybrid_clock.cc:648] HybridClock initialized: now 1790333700319153 us; error 57 us; skew 500 ppm
I20260925 10:55:00.319562 6158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:00 dist-test-slave-nx7s krb5kdc[5652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333700, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:00.322835 6158 init.cc:377] Logged in from keytab as oryx/127.31.62.189@KRBTEST.COM (short username oryx)
I20260925 10:55:00.324445 6158 webserver.cc:460] Webserver started at http://127.31.62.189:37149/ using document root <none> and password file <none>
I20260925 10:55:00.325224 6158 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:00.325341 6158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:00.325644 6158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:00.328732 6158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/data/instance:
uuid: "089603cb4c1d4455b174b99ade00cdf7"
format_stamp: "Formatted at 2026-09-25 10:55:00 on dist-test-slave-nx7s"
server_key: "17d314ad6d4c668e29b4d0131614be54"
server_key_iv: "cd849a390067e3150b4870b3f2aeea54"
server_key_version: "encryptionkey@0"
I20260925 10:55:00.329589 6158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/wal/instance:
uuid: "089603cb4c1d4455b174b99ade00cdf7"
format_stamp: "Formatted at 2026-09-25 10:55:00 on dist-test-slave-nx7s"
server_key: "17d314ad6d4c668e29b4d0131614be54"
server_key_iv: "cd849a390067e3150b4870b3f2aeea54"
server_key_version: "encryptionkey@0"
I20260925 10:55:00.334980 6158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:55:00.338687 6173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:00.340148 6158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:55:00.340332 6158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/wal
uuid: "089603cb4c1d4455b174b99ade00cdf7"
format_stamp: "Formatted at 2026-09-25 10:55:00 on dist-test-slave-nx7s"
server_key: "17d314ad6d4c668e29b4d0131614be54"
server_key_iv: "cd849a390067e3150b4870b3f2aeea54"
server_key_version: "encryptionkey@0"
I20260925 10:55:00.340543 6158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:00.358860 6158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:00.363197 6158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:00.363461 6158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:00.381400 6158 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.189:36481
I20260925 10:55:00.381429 6224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.189:36481 every 8 connection(s)
I20260925 10:55:00.382728 6158 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/data/info.pb
I20260925 10:55:00.386595 6225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:00.390072 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 6158
I20260925 10:55:00.393114 6020 heartbeater.cc:503] Master 127.31.62.190:40377 was elected leader, sending a full tablet report...
I20260925 10:55:00.394765 6225 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 10:55:00 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.62.189@KRBTEST.COM
Sep 25 10:55:00 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333700, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.189@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
I20260925 10:55:00.411587 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:00.397963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35264 (local address 127.31.62.190:40377)
0925 10:55:00.398409 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:00.398415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:00.398835 (+ 420us) server_negotiation.cc:407] Connection header received
0925 10:55:00.399005 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:00.399010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:00.399072 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:00.399157 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:00.400258 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.401058 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.402393 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.402769 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.402870 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:00.406219 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:00.406223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:00.406226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:00.406260 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:00.408880 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:00.409564 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:00.409569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:00.409571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:00.409690 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:00.410047 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:00.410053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:00.410056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:00.410413 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:00.410553 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:00.411080 (+ 527us) server_negotiation.cc:299] Negotiation successful
0925 10:55:00.411298 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":220,"threads_started":1}
I20260925 10:55:00.412994 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:00.392505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52004 (local address 127.31.62.189:36481)
0925 10:55:00.393482 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:00.393496 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:00.393543 (+ 47us) server_negotiation.cc:407] Connection header received
0925 10:55:00.395428 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:00.395459 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:00.395922 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:00.396304 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:00.397864 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.399612 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.400843 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.401570 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.401746 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:00.405796 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:00.405804 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:00.405818 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:00.405852 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:00.408710 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:00.409437 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:00.409444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:00.409447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:00.409601 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:00.409972 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:00.409976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:00.409978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:00.410539 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:00.410816 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:00.411149 (+ 333us) server_negotiation.cc:299] Negotiation successful
0925 10:55:00.411828 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":391,"threads_started":1}
I20260925 10:55:00.413014 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:00.397932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:35264)
0925 10:55:00.398704 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:00.398740 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:00.398891 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:00.399239 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:00.399246 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:00.399887 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:00.400116 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:00.400124 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.401197 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.401200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:00.402262 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:00.402274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.402968 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.402975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:00.403122 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:00.403694 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:00.403715 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:00.406017 (+ 2302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:00.409002 (+ 2985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:00.409014 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:00.409028 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:00.409444 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:00.409789 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:00.409794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:00.409798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:00.409947 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:00.410537 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:00.410545 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:00.410740 (+ 195us) client_negotiation.cc:769] Sending connection context
0925 10:55:00.410981 (+ 241us) client_negotiation.cc:240] Negotiation successful
0925 10:55:00.411963 (+ 982us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":445,"thread_start_us":430,"threads_started":1}
I20260925 10:55:00.415463 6225 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 10:55:00 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333700, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.189@KRBTEST.COM for oryx/127.31.62.189@KRBTEST.COM
I20260925 10:55:00.427647 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:00.416307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:36481 (local address 127.0.0.1:52020)
0925 10:55:00.416511 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:00.416525 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:00.416612 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:00.416891 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:00.416895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:00.417190 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:00.417328 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:00.417336 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.418420 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.418425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:00.419460 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:00.419472 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.420156 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.420160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:00.420314 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:00.420838 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:00.420903 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:00.422829 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:00.425539 (+ 2710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:00.425543 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:00.425545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:00.425894 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:00.426329 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:00.426332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:00.426334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:00.426460 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:00.426921 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:00.426924 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:00.427056 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 10:55:00.427164 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 10:55:00.427411 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123}
I20260925 10:55:00.427837 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:00.416268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52020 (local address 127.31.62.189:36481)
0925 10:55:00.416431 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:00.416435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:00.416607 (+ 172us) server_negotiation.cc:407] Connection header received
0925 10:55:00.416700 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:00.416704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:00.416761 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:00.416870 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:00.417462 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.418265 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.419615 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.420013 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.420116 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:00.423003 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:00.423007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:00.423011 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:00.423038 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:00.425423 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:00.426008 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:00.426012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:00.426014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:00.426193 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:00.426541 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:00.426544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:00.426546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:00.426819 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:00.426929 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:00.427257 (+ 328us) server_negotiation.cc:299] Negotiation successful
0925 10:55:00.427567 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":124,"server-negotiator.queue_time_us":59}
I20260925 10:55:00.432667 6225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: Bootstrap starting.
I20260925 10:55:00.435984 6225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:00.437106 6225 log.cc:824] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:00.439982 6225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: No bootstrap required, opened a new log
I20260925 10:55:00.447360 6225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } }
I20260925 10:55:00.447710 6225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:00.447782 6225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089603cb4c1d4455b174b99ade00cdf7, State: Initialized, Role: FOLLOWER
I20260925 10:55:00.448375 6225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } }
I20260925 10:55:00.449231 6232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } } }
I20260925 10:55:00.449544 6232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 10:55:00.450304 6225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: configured and running, proceeding with master startup.
W20260925 10:55:00.458871 6243 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 10:55:00.459008 6243 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 10:55:00.462095 6158 master_runner.cc:428] Detected that this master 089603cb4c1d4455b174b99ade00cdf7 is joining an existing cluster
I20260925 10:55:00.462193 6158 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.62.189:36481
I20260925 10:55:00.493803 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:00.465925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35280 (local address 127.31.62.190:40377)
0925 10:55:00.466116 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:00.466122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:00.473638 (+ 7516us) server_negotiation.cc:407] Connection header received
0925 10:55:00.473916 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:00.473921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:00.474008 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:00.474069 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:00.476406 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.477525 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.479633 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.480152 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.480254 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:00.486888 (+ 6634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:00.486894 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:00.486899 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:00.486937 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:00.490557 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:00.491544 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:00.491549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:00.491552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:00.491688 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:00.492137 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:00.492142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:00.492145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:00.492549 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:00.492666 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:00.493280 (+ 614us) server_negotiation.cc:299] Negotiation successful
0925 10:55:00.493499 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 10:55:00.493839 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:00.465995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52036 (local address 127.31.62.189:36481)
0925 10:55:00.466294 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:00.466299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:00.473700 (+ 7401us) server_negotiation.cc:407] Connection header received
0925 10:55:00.473915 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:00.473920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:00.473992 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:00.474058 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:00.475782 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.476772 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.478215 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.478741 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.478848 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:00.486870 (+ 8022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:00.486878 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:00.486882 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:00.486923 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:00.490557 (+ 3634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:00.491522 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:00.491526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:00.491529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:00.491677 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:00.492124 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:00.492129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:00.492131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:00.492535 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:00.492667 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:00.493216 (+ 549us) server_negotiation.cc:299] Negotiation successful
0925 10:55:00.493448 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":163}
I20260925 10:55:00.494750 6249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:00.466325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:36481 (local address 127.0.0.1:52036)
0925 10:55:00.473579 (+ 7254us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:00.473599 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:00.473795 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:00.474250 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:00.474254 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:00.475217 (+ 963us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:00.475587 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:00.475601 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.476982 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.476987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:00.478082 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:00.478097 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.478917 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.478923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:00.479102 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:00.485239 (+ 6137us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:00.485306 (+ 67us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:00.486611 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:00.490733 (+ 4122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:00.490738 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:00.490742 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:00.491399 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:00.491821 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:00.491826 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:00.491830 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:00.492017 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:00.492747 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:00.492751 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:00.492957 (+ 206us) client_negotiation.cc:769] Sending connection context
0925 10:55:00.494019 (+ 1062us) client_negotiation.cc:240] Negotiation successful
0925 10:55:00.494303 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7105,"mutex_wait_us":3,"spinlock_wait_cycles":61440,"thread_start_us":6541,"threads_started":1}
I20260925 10:55:00.495589 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:00.465974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:35280)
0925 10:55:00.473496 (+ 7522us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:00.473519 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:00.473767 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:00.474281 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:00.474285 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:00.475844 (+ 1559us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:00.476178 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:00.476190 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.477756 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.477761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:00.479495 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:00.479509 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.480339 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.480346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:00.480528 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:00.485239 (+ 4711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:00.485278 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:00.486611 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:00.490733 (+ 4122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:00.490737 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:00.490742 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:00.491402 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:00.491821 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:00.491826 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:00.491830 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:00.492020 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:00.492747 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:00.492751 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:00.493003 (+ 252us) client_negotiation.cc:769] Sending connection context
0925 10:55:00.494526 (+ 1523us) client_negotiation.cc:240] Negotiation successful
0925 10:55:00.495142 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7346,"mutex_wait_us":71,"spinlock_wait_cycles":11904,"thread_start_us":6875,"threads_started":1}
WARNING: no policy specified for oryx/127.31.62.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.62.188@KRBTEST.COM" created.
Sep 25 10:55:00 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333698, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.190@KRBTEST.COM for oryx/127.31.62.189@KRBTEST.COM
I20260925 10:55:00.521984 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:00.503218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52042 (local address 127.31.62.189:36481)
0925 10:55:00.503390 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:00.503396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:00.505566 (+ 2170us) server_negotiation.cc:407] Connection header received
0925 10:55:00.505971 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:00.505977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:00.506093 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:00.506195 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:00.507621 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.508579 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.510538 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.511170 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.511283 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:00.515437 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:00.515443 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:00.515448 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:00.515489 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:00.519140 (+ 3651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:00.519936 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:00.519940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:00.519942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:00.520078 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:00.520451 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:00.520456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:00.520459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:00.520752 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:00.520926 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:00.521389 (+ 463us) server_negotiation.cc:299] Negotiation successful
0925 10:55:00.521599 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 10:55:00.522030 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:00.503163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:36481 (local address 127.0.0.1:52042)
0925 10:55:00.505423 (+ 2260us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:00.505476 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:00.505737 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:00.506273 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:00.506281 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:00.507153 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:00.507441 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:00.507453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.508721 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.508727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:00.510334 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:00.510349 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.511339 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.511346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:00.511547 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:00.512282 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:00.512306 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:00.515215 (+ 2909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:00.519300 (+ 4085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:00.519314 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:00.519320 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:00.519747 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:00.520187 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:00.520192 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:00.520194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:00.520334 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:00.520903 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:00.520912 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:00.521085 (+ 173us) client_negotiation.cc:769] Sending connection context
0925 10:55:00.521258 (+ 173us) client_negotiation.cc:240] Negotiation successful
0925 10:55:00.521599 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1996,"thread_start_us":1838,"threads_started":1}
I20260925 10:55:00.524011 5697 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.62.189:36481: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7"
member_type: NON_VOTER
last_known_addr {
host: "127.31.62.189"
port: 36481
}
attrs {
promote: true
}
}
dest_uuid: "44235815436447c3ab94d41adcbe92c5"
cas_config_opid_index: -1
I20260925 10:55:00.525228 5697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: NON_VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.62.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.188.keytab.
Entry for principal oryx/127.31.62.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.188.keytab.
I20260925 10:55:00.528409 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.188:46577
--webserver_interface=127.31.62.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.188.keytab
--principal=oryx/127.31.62.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:33393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:40377,127.31.62.189:36481,127.31.62.188:46577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:55:00.536142 6200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:00.538868 6255 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 089603cb4c1d4455b174b99ade00cdf7 (127.31.62.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: NON_VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: true } } }
I20260925 10:55:00.539880 6200 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 FOLLOWER]: Refusing update from remote peer 44235815436447c3ab94d41adcbe92c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260925 10:55:00.543785 6259 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: NON_VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:00.544375 6255 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.62.189:36481
I20260925 10:55:00.545225 6158 master.cc:537] Master@127.31.62.189:36481 shutting down...
I20260925 10:55:00.548133 6255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: NON_VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: true } } }
I20260925 10:55:00.548565 6255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: This master's current role is: LEADER
W20260925 10:55:00.556864 5682 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260925 10:55:00.557866 6158 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:00.558331 6158 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:00.558525 6158 tablet_replica.cc:330] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: stopping tablet replica
W20260925 10:55:00.558807 5682 consensus_peers.cc:487] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 -> Peer 089603cb4c1d4455b174b99ade00cdf7 (127.31.62.189:36481): Couldn't send request to peer 089603cb4c1d4455b174b99ade00cdf7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 10:55:00.575905 6158 master.cc:552] Master@127.31.62.189:36481 shutdown complete.
I20260925 10:55:00.576009 6158 master_runner.cc:305] Clearing existing system tablet
I20260925 10:55:00.576480 6158 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:00.576584 6158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:00.579916 6158 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 10:55:00.581976 6263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:00.582414 6158 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:00.582690 6158 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 10:55:00.582774 6158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/wal
uuid: "089603cb4c1d4455b174b99ade00cdf7"
format_stamp: "Formatted at 2026-09-25 10:55:00 on dist-test-slave-nx7s"
server_key: "17d314ad6d4c668e29b4d0131614be54"
server_key_iv: "cd849a390067e3150b4870b3f2aeea54"
server_key_version: "encryptionkey@0"
I20260925 10:55:00.587927 6158 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:00.595906 6158 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 10:55:00.596156 6158 log.cc:1196] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260925 10:55:00.596626 6158 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: Deleting consensus metadata
I20260925 10:55:00.596908 6158 master_runner.cc:315] Copying system tablet from 127.31.62.190:40377
I20260925 10:55:00.602237 6158 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: tablet copy: Beginning tablet copy session from remote peer at address 127.31.62.190:40377
I20260925 10:55:00.619750 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:00.603088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:35288)
0925 10:55:00.605367 (+ 2279us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:00.605393 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:00.605590 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:00.605985 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:00.605990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:00.606488 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:00.606825 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:00.606839 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.608295 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.608300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:00.609873 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:00.609888 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.610875 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.610881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:00.611062 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:00.611773 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:00.611795 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:00.612824 (+ 1029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:00.616765 (+ 3941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:00.616771 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:00.616776 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:00.617342 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:00.617769 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:00.617773 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:00.617777 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:00.617963 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:00.618575 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:00.618580 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:00.618808 (+ 228us) client_negotiation.cc:769] Sending connection context
0925 10:55:00.618969 (+ 161us) client_negotiation.cc:240] Negotiation successful
0925 10:55:00.619289 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2134,"thread_start_us":1964,"threads_started":1}
I20260925 10:55:00.620093 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:00.603133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35288 (local address 127.31.62.190:40377)
0925 10:55:00.603325 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:00.603331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:00.605528 (+ 2197us) server_negotiation.cc:407] Connection header received
0925 10:55:00.605711 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:00.605716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:00.605826 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:00.605943 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:00.606984 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.608149 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.610030 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.610720 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.610840 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:00.613046 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:00.613053 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:00.613058 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:00.613097 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:00.616514 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:00.617483 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:00.617489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:00.617493 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:00.617653 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:00.618071 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:00.618081 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:00.618085 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:00.618448 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:00.618592 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:00.619105 (+ 513us) server_negotiation.cc:299] Negotiation successful
0925 10:55:00.619330 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 10:55:00.620824 5717 tablet_copy_service.cc:140] P 44235815436447c3ab94d41adcbe92c5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 089603cb4c1d4455b174b99ade00cdf7 ({username='oryx', principal='oryx/127.31.62.189@KRBTEST.COM'} at 127.0.0.1:35288)
I20260925 10:55:00.621157 5717 tablet_copy_service.cc:161] P 44235815436447c3ab94d41adcbe92c5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 089603cb4c1d4455b174b99ade00cdf7 at {username='oryx', principal='oryx/127.31.62.189@KRBTEST.COM'} at 127.0.0.1:35288: session id = 089603cb4c1d4455b174b99ade00cdf7-00000000000000000000000000000000
I20260925 10:55:00.625473 5717 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 10:55:00.627733 6158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 10:55:00.634511 6158 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: tablet copy: Starting download of 0 data blocks...
I20260925 10:55:00.634860 6158 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: tablet copy: Starting download of 1 WAL segments...
I20260925 10:55:00.637753 6158 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 10:55:00.642428 5717 tablet_copy_service.cc:342] P 44235815436447c3ab94d41adcbe92c5: Request end of tablet copy session 089603cb4c1d4455b174b99ade00cdf7-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.62.189@KRBTEST.COM'} at 127.0.0.1:35288
I20260925 10:55:00.642660 5717 tablet_copy_service.cc:433] P 44235815436447c3ab94d41adcbe92c5: ending tablet copy session 089603cb4c1d4455b174b99ade00cdf7-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 089603cb4c1d4455b174b99ade00cdf7
I20260925 10:55:00.653043 6158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:00.655742 6270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:00.662077 6271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:00.666780 6158 server_base.cc:1034] running on GCE node
W20260925 10:55:00.667127 6273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:00.667543 6158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:00.668176 6158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:00.669412 6158 hybrid_clock.cc:648] HybridClock initialized: now 1790333700669350 us; error 38 us; skew 500 ppm
I20260925 10:55:00.669696 6158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:00 dist-test-slave-nx7s krb5kdc[5652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333700, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:00.671674 6158 init.cc:377] Logged in from keytab as oryx/127.31.62.189@KRBTEST.COM (short username oryx)
I20260925 10:55:00.673520 6158 webserver.cc:460] Webserver started at http://127.31.62.189:36979/ using document root <none> and password file <none>
I20260925 10:55:00.673818 6158 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:00.673894 6158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:00.676415 6158 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260925 10:55:00.678118 6280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:00.678514 6158 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 10:55:00.678642 6158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/wal
uuid: "089603cb4c1d4455b174b99ade00cdf7"
format_stamp: "Formatted at 2026-09-25 10:55:00 on dist-test-slave-nx7s"
server_key: "17d314ad6d4c668e29b4d0131614be54"
server_key_iv: "cd849a390067e3150b4870b3f2aeea54"
server_key_version: "encryptionkey@0"
I20260925 10:55:00.678781 6158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:00.697953 6158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:00.701283 6158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:00.723814 6158 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.189:36481
I20260925 10:55:00.723834 6331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.189:36481 every 8 connection(s)
I20260925 10:55:00.724650 6158 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-1/data/info.pb
I20260925 10:55:00.730108 6332 sys_catalog.cc:263] Verifying existing consensus state
I20260925 10:55:00.733436 6332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: Bootstrap starting.
I20260925 10:55:00.745869 6332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 10:55:00.746330 6332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: Bootstrap complete.
I20260925 10:55:00.747164 6332 raft_consensus.cc:348] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: NON_VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: true } }
I20260925 10:55:00.747382 6332 raft_consensus.cc:729] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 089603cb4c1d4455b174b99ade00cdf7, State: Initialized, Role: LEARNER
I20260925 10:55:00.747608 6332 consensus_queue.cc:260] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: NON_VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: true } }
I20260925 10:55:00.748404 6335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: NON_VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: true } } }
I20260925 10:55:00.748669 6332 sys_catalog.cc:564] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:55:00.748884 6335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: This master's current role is: LEARNER
I20260925 10:55:00.756583 6346 catalog_manager.cc:1259] Loaded cluster ID: 86416e3acf79441b9253cc5f1538d331
I20260925 10:55:00.756680 6346 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: loading cluster ID for follower catalog manager: success
I20260925 10:55:00.759289 6346 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: acquiring CA information for follower catalog manager: success
I20260925 10:55:00.761377 6346 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 10:55:00.896795 6153 heartbeater.cc:503] Master 127.31.62.190:40377 was elected leader, sending a full tablet report...
I20260925 10:55:00.966979 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:00.960940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:36481 (local address 127.0.0.1:52048)
0925 10:55:00.961395 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:00.961417 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:00.961550 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:00.961951 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:00.961955 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:00.961983 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:00.962162 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:00.962169 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.963442 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.963448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:00.964674 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:00.964688 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.966180 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.966186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:00.966409 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:00.966474 (+ 65us) client_negotiation.cc:769] Sending connection context
0925 10:55:00.966559 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 10:55:00.966713 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325}
I20260925 10:55:00.967021 6347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:00.961090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52048 (local address 127.31.62.189:36481)
0925 10:55:00.961616 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:00.961621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:00.961646 (+ 25us) server_negotiation.cc:407] Connection header received
0925 10:55:00.961713 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:00.961718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:00.961791 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:00.961894 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:00.962302 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.963298 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.964837 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:00.966012 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:00.966130 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:00.966380 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:00.966597 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 10:55:00.966729 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":539,"threads_started":1}
I20260925 10:55:00.968396 6307 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260925 10:55:00.968957 6335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44235815436447c3ab94d41adcbe92c5. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: NON_VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: true } } }
I20260925 10:55:00.969224 6335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: This master's current role is: LEARNER
W20260925 10:55:00.973345 6257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:00.973634 6257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:00.973755 6257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:00.982266 6257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:55:00.982386 6257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:00.982443 6257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:00.982519 6257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:55:00.982582 6257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:55:00.993672 6257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:33393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:40377,127.31.62.189:36481,127.31.62.188:46577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.188.keytab
--principal=oryx/127.31.62.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.188:46577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.62.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:00.995412 6257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:00.996908 6257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:01.009225 6352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:01.009320 6353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:01.009369 6355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:01.010576 6257 server_base.cc:1034] running on GCE node
I20260925 10:55:01.011554 6257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:01.013269 6257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:01.014509 6257 hybrid_clock.cc:648] HybridClock initialized: now 1790333701014464 us; error 43 us; skew 500 ppm
I20260925 10:55:01.014907 6257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:01 dist-test-slave-nx7s krb5kdc[5652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333701, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:01.018162 6257 init.cc:377] Logged in from keytab as oryx/127.31.62.188@KRBTEST.COM (short username oryx)
I20260925 10:55:01.019649 6257 webserver.cc:460] Webserver started at http://127.31.62.188:44683/ using document root <none> and password file <none>
I20260925 10:55:01.020311 6257 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:01.020413 6257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:01.020665 6257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:01.023650 6257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/data/instance:
uuid: "e6a2bc1541614ee3b1c9f430a8e74994"
format_stamp: "Formatted at 2026-09-25 10:55:01 on dist-test-slave-nx7s"
server_key: "d7ad9f2377a10730f069a278ab647c0c"
server_key_iv: "88b4fc83663665f69343f2d74414449a"
server_key_version: "encryptionkey@0"
I20260925 10:55:01.024386 6257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/wal/instance:
uuid: "e6a2bc1541614ee3b1c9f430a8e74994"
format_stamp: "Formatted at 2026-09-25 10:55:01 on dist-test-slave-nx7s"
server_key: "d7ad9f2377a10730f069a278ab647c0c"
server_key_iv: "88b4fc83663665f69343f2d74414449a"
server_key_version: "encryptionkey@0"
I20260925 10:55:01.029662 6257 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 10:55:01.033160 6362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:01.034597 6257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 10:55:01.034757 6257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/wal
uuid: "e6a2bc1541614ee3b1c9f430a8e74994"
format_stamp: "Formatted at 2026-09-25 10:55:01 on dist-test-slave-nx7s"
server_key: "d7ad9f2377a10730f069a278ab647c0c"
server_key_iv: "88b4fc83663665f69343f2d74414449a"
server_key_version: "encryptionkey@0"
I20260925 10:55:01.035377 6257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:01.052790 6257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:01.057404 6257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:01.057655 6257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:01.076399 6257 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.188:46577
I20260925 10:55:01.076449 6413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.188:46577 every 8 connection(s)
I20260925 10:55:01.077889 6257 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/data/info.pb
I20260925 10:55:01.082295 6414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:01.087383 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 6257
I20260925 10:55:01.089433 6414 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 10:55:01 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333701, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.188@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
Sep 25 10:55:01 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.62.188@KRBTEST.COM
I20260925 10:55:01.109447 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.092508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35294 (local address 127.31.62.190:40377)
0925 10:55:01.092697 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:01.092703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:01.093451 (+ 748us) server_negotiation.cc:407] Connection header received
0925 10:55:01.093802 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:01.093808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:01.093879 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:01.093972 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:01.095544 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.096604 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.098798 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.099348 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.099474 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.103297 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:01.103302 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:01.103305 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:01.103337 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:01.106160 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.106939 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.106944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.106948 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.107091 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.107489 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.107494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.107497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.107822 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:01.107923 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:01.108941 (+ 1018us) server_negotiation.cc:299] Negotiation successful
0925 10:55:01.109110 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 10:55:01.110615 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.092464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:35294)
0925 10:55:01.093271 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:01.093312 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:01.093578 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:01.094052 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:01.094058 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:01.094845 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:01.095353 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.095378 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.096782 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.096786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.098650 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.098664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.099564 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.099570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.099774 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.100527 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:01.100558 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:01.103101 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:01.106298 (+ 3197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.106308 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.106318 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.106809 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.107198 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.107201 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.107203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.107379 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.107986 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:01.107997 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:01.108465 (+ 468us) client_negotiation.cc:769] Sending connection context
0925 10:55:01.108810 (+ 345us) client_negotiation.cc:240] Negotiation successful
0925 10:55:01.109323 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":29,"thread_start_us":388,"threads_started":1}
I20260925 10:55:01.110952 6417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.090097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59108 (local address 127.31.62.188:46577)
0925 10:55:01.090937 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:01.090954 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:01.091011 (+ 57us) server_negotiation.cc:407] Connection header received
0925 10:55:01.093158 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:01.093182 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:01.093593 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:01.093821 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:01.095551 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.097103 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.098926 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.099752 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.099908 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.104540 (+ 4632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:01.104551 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:01.104570 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:01.104605 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:01.107630 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.108531 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.108537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.108539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.108668 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.109213 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.109219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.109222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.109631 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:01.109905 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:01.110281 (+ 376us) server_negotiation.cc:299] Negotiation successful
0925 10:55:01.110519 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":599,"thread_start_us":361,"threads_started":1}
I20260925 10:55:01.112450 6414 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 10:55:01 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333701, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.188@KRBTEST.COM for oryx/127.31.62.189@KRBTEST.COM
I20260925 10:55:01.125363 6347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.113327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52064 (local address 127.31.62.189:36481)
0925 10:55:01.113497 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:01.113504 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:01.113521 (+ 17us) server_negotiation.cc:407] Connection header received
0925 10:55:01.113634 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:01.113640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:01.113715 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:01.113820 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:01.114446 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.115175 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.116423 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.116837 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.116962 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.120224 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:01.120229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:01.120232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:01.120264 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:01.122852 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.123567 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.123571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.123573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.123708 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.124149 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.124152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.124154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.124417 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:01.124531 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:01.124954 (+ 423us) server_negotiation.cc:299] Negotiation successful
0925 10:55:01.125126 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 10:55:01.125408 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.113208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:36481 (local address 127.0.0.1:52064)
0925 10:55:01.113380 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:01.113399 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:01.113495 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:01.113871 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:01.113875 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:01.114189 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:01.114314 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.114322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.115317 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.115321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.116299 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.116310 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.117001 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.117005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.117178 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.117718 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:01.117734 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:01.119982 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:01.122993 (+ 3011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.122998 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.123000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.123426 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.123831 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.123835 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.123838 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.123989 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.124530 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:01.124534 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:01.124686 (+ 152us) client_negotiation.cc:769] Sending connection context
0925 10:55:01.124819 (+ 133us) client_negotiation.cc:240] Negotiation successful
0925 10:55:01.125164 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 10:55:01.126412 6414 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 10:55:01 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333701, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.188@KRBTEST.COM for oryx/127.31.62.188@KRBTEST.COM
I20260925 10:55:01.137626 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.127177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:46577 (local address 127.0.0.1:59116)
0925 10:55:01.127325 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:01.127340 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:01.127427 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:01.127693 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:01.127697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:01.128013 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:01.128189 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.128197 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.129094 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.129098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.129898 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.129915 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.130504 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.130509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.130699 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.131231 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:01.131247 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:01.133032 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:01.135477 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.135483 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.135487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.135795 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.136093 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.136098 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.136102 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.136235 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.136676 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:01.136680 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:01.136800 (+ 120us) client_negotiation.cc:769] Sending connection context
0925 10:55:01.136958 (+ 158us) client_negotiation.cc:240] Negotiation successful
0925 10:55:01.137204 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260925 10:55:01.137633 6417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.127344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59116 (local address 127.31.62.188:46577)
0925 10:55:01.127470 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:01.127476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:01.127490 (+ 14us) server_negotiation.cc:407] Connection header received
0925 10:55:01.127526 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:01.127529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:01.127573 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:01.127667 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:01.128303 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.128991 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.130018 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.130386 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.130484 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.133196 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:01.133200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:01.133203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:01.133227 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:01.135371 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.135888 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.135893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.135903 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.136003 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.136323 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.136327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.136331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.136568 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:01.136689 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:01.137035 (+ 346us) server_negotiation.cc:299] Negotiation successful
0925 10:55:01.137348 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":117,"server-negotiator.queue_time_us":42}
I20260925 10:55:01.142324 6414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: Bootstrap starting.
I20260925 10:55:01.145910 6414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:01.146894 6414 log.cc:824] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:01.149559 6414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: No bootstrap required, opened a new log
I20260925 10:55:01.156330 6414 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } }
I20260925 10:55:01.156631 6414 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:01.156711 6414 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6a2bc1541614ee3b1c9f430a8e74994, State: Initialized, Role: FOLLOWER
I20260925 10:55:01.157248 6414 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } }
I20260925 10:55:01.158012 6420 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } } }
I20260925 10:55:01.158321 6420 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 10:55:01.159281 6414 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:55:01.165584 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 5758
W20260925 10:55:01.168322 6431 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 10:55:01.168552 6431 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 10:55:01.172405 6257 master_runner.cc:428] Detected that this master e6a2bc1541614ee3b1c9f430a8e74994 is joining an existing cluster
I20260925 10:55:01.172534 6257 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.62.188:46577
Sep 25 10:55:01 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.62.129@KRBTEST.COM
I20260925 10:55:01.195341 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.167264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44470 (local address 127.31.62.129:34941)
0925 10:55:01.169148 (+ 1884us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:01.169157 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:01.169187 (+ 30us) server_negotiation.cc:407] Connection header received
0925 10:55:01.169255 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:01.169264 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:01.169552 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:01.169716 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:01.171258 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.172409 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.173831 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.174841 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.174947 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.185524 (+ 10577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:01.185547 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:01.185570 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:01.185629 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:01.191196 (+ 5567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.192311 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.192322 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.192325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.192571 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.193178 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.193183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.193187 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.193644 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:01.193932 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:01.194622 (+ 690us) server_negotiation.cc:299] Negotiation successful
0925 10:55:01.194911 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1700,"thread_start_us":1574,"threads_started":1}
I20260925 10:55:01.197865 5881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:01.209931 6347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.177771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52080 (local address 127.31.62.189:36481)
0925 10:55:01.178140 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:01.178145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:01.190253 (+ 12108us) server_negotiation.cc:407] Connection header received
0925 10:55:01.190455 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:01.190461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:01.190556 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:01.190681 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:01.192196 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.193301 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.195374 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.195988 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.196089 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.201084 (+ 4995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:01.201092 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:01.201097 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:01.201221 (+ 124us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:01.206430 (+ 5209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.207365 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.207370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.207374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.207534 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.208041 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.208046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.208049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.208423 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:01.208542 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:01.209300 (+ 758us) server_negotiation.cc:299] Negotiation successful
0925 10:55:01.209565 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236}
I20260925 10:55:01.211122 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.177603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:36481 (local address 127.0.0.1:52080)
0925 10:55:01.190082 (+ 12479us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:01.190111 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:01.190316 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:01.190895 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:01.190901 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:01.191554 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:01.191993 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.192008 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.193492 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.193497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.195197 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.195211 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.196658 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.196665 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.196877 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.198942 (+ 2065us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:01.198963 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:01.199991 (+ 1028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:01.206634 (+ 6643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.206640 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.206644 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.207212 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.207685 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.207690 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.207694 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.207903 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.208637 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:01.208643 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:01.208979 (+ 336us) client_negotiation.cc:769] Sending connection context
0925 10:55:01.210218 (+ 1239us) client_negotiation.cc:240] Negotiation successful
0925 10:55:01.210626 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12236,"mutex_wait_us":55,"thread_start_us":11978,"threads_started":1}
I20260925 10:55:01.212962 6439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.177882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:46577 (local address 127.0.0.1:59120)
0925 10:55:01.190891 (+ 13009us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:01.190907 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:01.191015 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:01.191535 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:01.191540 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:01.192238 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:01.192651 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.192664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.194340 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.194349 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.195971 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.195985 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.197324 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.197331 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.198144 (+ 813us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.204230 (+ 6086us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:01.204277 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:01.205454 (+ 1177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:01.209538 (+ 4084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.209543 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.209547 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.210346 (+ 799us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.210755 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.210760 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.210763 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.210916 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.211918 (+ 1002us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:01.211921 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:01.212053 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 10:55:01.212211 (+ 158us) client_negotiation.cc:240] Negotiation successful
0925 10:55:01.212517 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12889,"mutex_wait_us":2,"thread_start_us":12683,"threads_started":1}
I20260925 10:55:01.213100 6417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.178275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59120 (local address 127.31.62.188:46577)
0925 10:55:01.178411 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:01.178417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:01.190992 (+ 12575us) server_negotiation.cc:407] Connection header received
0925 10:55:01.191105 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:01.191109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:01.191173 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:01.191264 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:01.192815 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.194126 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.196138 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.196911 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.197002 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.205631 (+ 8629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:01.205637 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:01.205641 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:01.205682 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:01.209391 (+ 3709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.210466 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.210470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.210474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.210623 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.211016 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.211020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.211023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.211731 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:01.211844 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:01.212333 (+ 489us) server_negotiation.cc:299] Negotiation successful
0925 10:55:01.212681 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":113,"server-negotiator.queue_time_us":30}
I20260925 10:55:01.219044 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.177497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35306 (local address 127.31.62.190:40377)
0925 10:55:01.177982 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:01.177988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:01.198657 (+ 20669us) server_negotiation.cc:407] Connection header received
0925 10:55:01.198879 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:01.198884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:01.198964 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:01.199049 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:01.201482 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.202976 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.207297 (+ 4321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.207940 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.208049 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.213732 (+ 5683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:01.213739 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:01.213743 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:01.213788 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:01.216387 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.217094 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.217099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.217101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.217231 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.217592 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.217597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.217599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.217878 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:01.218027 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:01.218489 (+ 462us) server_negotiation.cc:299] Negotiation successful
0925 10:55:01.218706 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352}
I20260925 10:55:01.219118 6440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.178622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:35306)
0925 10:55:01.198540 (+ 19918us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:01.198559 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:01.198766 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:01.200158 (+ 1392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:01.200163 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:01.200721 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:01.201236 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.201250 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.203210 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.203220 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.207093 (+ 3873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.207109 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.208342 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.208349 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.208554 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.209248 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:01.209274 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:01.211845 (+ 2571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:01.216527 (+ 4682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.216533 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.216537 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.216971 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.217334 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.217339 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.217343 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.217482 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.217997 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:01.218002 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:01.218198 (+ 196us) client_negotiation.cc:769] Sending connection context
0925 10:55:01.218311 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 10:55:01.218636 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19791,"thread_start_us":19629,"threads_started":1}
Sep 25 10:55:01 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333698, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.190@KRBTEST.COM for oryx/127.31.62.188@KRBTEST.COM
I20260925 10:55:01.243530 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.224207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:46577 (local address 127.0.0.1:59130)
0925 10:55:01.224521 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:01.224538 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:01.224635 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:01.225550 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:01.225555 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:01.226028 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:01.226242 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.226254 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.227975 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.227981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.229448 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.229464 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.231271 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.231280 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.231499 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.232243 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:01.232273 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:01.235205 (+ 2932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:01.240531 (+ 5326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.240538 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.240542 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.241140 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.241629 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.241634 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.241639 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.241791 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.242459 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:01.242464 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:01.242642 (+ 178us) client_negotiation.cc:769] Sending connection context
0925 10:55:01.242764 (+ 122us) client_negotiation.cc:240] Negotiation successful
0925 10:55:01.243111 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260925 10:55:01.244617 6417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.224843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59130 (local address 127.31.62.188:46577)
0925 10:55:01.225040 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:01.225045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:01.225062 (+ 17us) server_negotiation.cc:407] Connection header received
0925 10:55:01.225168 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:01.225173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:01.225277 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:01.225361 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:01.226450 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.227814 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.229613 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.230603 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.230750 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.235477 (+ 4727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:01.235485 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:01.235489 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:01.235552 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:01.240313 (+ 4761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.241338 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.241344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.241348 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.241498 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.241923 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.241928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.241932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.242326 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:01.243916 (+ 1590us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:01.244040 (+ 124us) server_negotiation.cc:299] Negotiation successful
0925 10:55:01.244303 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 10:55:01.246181 5697 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.62.188:46577: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994"
member_type: NON_VOTER
last_known_addr {
host: "127.31.62.188"
port: 46577
}
attrs {
promote: true
}
}
dest_uuid: "44235815436447c3ab94d41adcbe92c5"
cas_config_opid_index: 5
I20260925 10:55:01.247339 5697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: NON_VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: true } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } }
I20260925 10:55:01.249554 6259 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e6a2bc1541614ee3b1c9f430a8e74994 (127.31.62.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: NON_VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: true } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } } }
I20260925 10:55:01.253964 6307 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 LEARNER]: Refusing update from remote peer 44235815436447c3ab94d41adcbe92c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 10:55:01.256229 6389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:01.260648 6389 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 1 FOLLOWER]: Refusing update from remote peer 44235815436447c3ab94d41adcbe92c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 10:55:01.259898 6442 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: NON_VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260925 10:55:01.261688 6259 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.62.188:46577
I20260925 10:55:01.262455 6257 master.cc:537] Master@127.31.62.188:46577 shutting down...
I20260925 10:55:01.264299 6259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: NON_VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: true } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } } }
I20260925 10:55:01.264670 6259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:01.265087 6443 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:01.271309 6307 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e6a2bc1541614ee3b1c9f430a8e74994 (127.31.62.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: NON_VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: true } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } } }
I20260925 10:55:01.275985 6335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: NON_VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: true } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } } }
I20260925 10:55:01.276283 6335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: This master's current role is: LEARNER
I20260925 10:55:01.277408 6445 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5: attempting to promote NON_VOTER 089603cb4c1d4455b174b99ade00cdf7 to VOTER
I20260925 10:55:01.278246 6445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } }
I20260925 10:55:01.286516 6307 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 LEARNER]: Refusing update from remote peer 44235815436447c3ab94d41adcbe92c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 10:55:01.287277 6445 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 10:55:01.292778 6259 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 089603cb4c1d4455b174b99ade00cdf7 (127.31.62.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } } }
I20260925 10:55:01.294775 6257 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:01.295208 6257 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:01.295418 6257 tablet_replica.cc:330] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: stopping tablet replica
I20260925 10:55:01.296713 6443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } } }
I20260925 10:55:01.297068 6443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:01.297572 6443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } } }
I20260925 10:55:01.297816 6443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:01.298924 6307 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 089603cb4c1d4455b174b99ade00cdf7 (127.31.62.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } } }
I20260925 10:55:01.302372 6257 master.cc:552] Master@127.31.62.188:46577 shutdown complete.
I20260925 10:55:01.302456 6257 master_runner.cc:305] Clearing existing system tablet
I20260925 10:55:01.302831 6257 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:01.302944 6257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:01.304450 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.303972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:46577 (local address 127.0.0.1:59132)
0925 10:55:01.304159 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:01.304278 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 10:55:01.304494 6335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } } }
I20260925 10:55:01.304785 6335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 10:55:01.304945 5681 consensus_peers.cc:487] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 -> Peer e6a2bc1541614ee3b1c9f430a8e74994 (127.31.62.188:46577): Couldn't send request to peer e6a2bc1541614ee3b1c9f430a8e74994. Status: Network error: Client connection negotiation failed: client connection to 127.31.62.188:46577: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 10:55:01.310150 6257 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.008s sys 0.000s
I20260925 10:55:01.312038 6449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:01.312363 6257 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:01.312623 6257 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 10:55:01.312717 6257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/wal
uuid: "e6a2bc1541614ee3b1c9f430a8e74994"
format_stamp: "Formatted at 2026-09-25 10:55:01 on dist-test-slave-nx7s"
server_key: "d7ad9f2377a10730f069a278ab647c0c"
server_key_iv: "88b4fc83663665f69343f2d74414449a"
server_key_version: "encryptionkey@0"
I20260925 10:55:01.317826 6257 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:01.324867 6257 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 10:55:01.325047 6257 log.cc:1196] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260925 10:55:01.325495 6257 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: Deleting consensus metadata
I20260925 10:55:01.325735 6257 master_runner.cc:315] Copying system tablet from 127.31.62.190:40377
I20260925 10:55:01.330411 6257 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: tablet copy: Beginning tablet copy session from remote peer at address 127.31.62.190:40377
I20260925 10:55:01.342195 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 5758
I20260925 10:55:01.346203 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.331341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35314 (local address 127.31.62.190:40377)
0925 10:55:01.331579 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:01.331586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:01.332107 (+ 521us) server_negotiation.cc:407] Connection header received
0925 10:55:01.332271 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:01.332277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:01.332363 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:01.332467 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:01.333512 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.334714 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.336396 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.337062 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.337197 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.339442 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:01.339448 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:01.339453 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:01.339493 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:01.343577 (+ 4084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.344277 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.344283 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.344287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.344420 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.344791 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.344795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.344798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.345126 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:01.345261 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:01.345653 (+ 392us) server_negotiation.cc:299] Negotiation successful
0925 10:55:01.345852 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260925 10:55:01.346259 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.331458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:35314)
0925 10:55:01.331978 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:01.332000 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:01.332151 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:01.332519 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:01.332524 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:01.333069 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:01.333315 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.333331 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.334880 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.334885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.336236 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.336252 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.337235 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.337242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.337431 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.338162 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:01.338185 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:01.339215 (+ 1030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:01.343737 (+ 4522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.343743 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.343747 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.344171 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.344523 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.344528 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.344532 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.344677 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.345231 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:01.345235 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:01.345414 (+ 179us) client_negotiation.cc:769] Sending connection context
0925 10:55:01.345533 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 10:55:01.345855 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":275,"threads_started":1}
I20260925 10:55:01.347049 5717 tablet_copy_service.cc:140] P 44235815436447c3ab94d41adcbe92c5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e6a2bc1541614ee3b1c9f430a8e74994 ({username='oryx', principal='oryx/127.31.62.188@KRBTEST.COM'} at 127.0.0.1:35314)
I20260925 10:55:01.347203 5717 tablet_copy_service.cc:161] P 44235815436447c3ab94d41adcbe92c5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e6a2bc1541614ee3b1c9f430a8e74994 at {username='oryx', principal='oryx/127.31.62.188@KRBTEST.COM'} at 127.0.0.1:35314: session id = e6a2bc1541614ee3b1c9f430a8e74994-00000000000000000000000000000000
I20260925 10:55:01.349813 5717 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 10:55:01.351660 6257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 10:55:01.358701 6257 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: tablet copy: Starting download of 0 data blocks...
I20260925 10:55:01.359030 6257 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: tablet copy: Starting download of 1 WAL segments...
I20260925 10:55:01.362668 6257 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 10:55:01.363183 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:34941
--local_ip_for_outbound_sockets=127.31.62.129
--tserver_master_addrs=127.31.62.190:40377
--webserver_port=40919
--webserver_interface=127.31.62.129
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.129.keytab
--principal=oryx/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:33393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.62.190:40377,127.31.62.189:36481
--tserver_master_addrs=127.31.62.190:40377,127.31.62.189:36481,127.31.62.188:46577 with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:55:01.366564 5717 tablet_copy_service.cc:342] P 44235815436447c3ab94d41adcbe92c5: Request end of tablet copy session e6a2bc1541614ee3b1c9f430a8e74994-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.62.188@KRBTEST.COM'} at 127.0.0.1:35314
I20260925 10:55:01.366715 5717 tablet_copy_service.cc:433] P 44235815436447c3ab94d41adcbe92c5: ending tablet copy session e6a2bc1541614ee3b1c9f430a8e74994-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e6a2bc1541614ee3b1c9f430a8e74994
I20260925 10:55:01.378403 6257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:01.381261 6457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:01.387821 6458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:01.391821 6460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:01.392529 6257 server_base.cc:1034] running on GCE node
I20260925 10:55:01.392936 6257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:01.393599 6257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:01.394857 6257 hybrid_clock.cc:648] HybridClock initialized: now 1790333701394815 us; error 36 us; skew 500 ppm
I20260925 10:55:01.395798 6257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:01 dist-test-slave-nx7s krb5kdc[5652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333701, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:01.398207 6257 init.cc:377] Logged in from keytab as oryx/127.31.62.188@KRBTEST.COM (short username oryx)
I20260925 10:55:01.403496 6257 webserver.cc:460] Webserver started at http://127.31.62.188:41671/ using document root <none> and password file <none>
I20260925 10:55:01.403931 6257 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:01.404085 6257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:01.407143 6257 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260925 10:55:01.409220 6467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:01.409718 6257 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260925 10:55:01.409907 6257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/wal
uuid: "e6a2bc1541614ee3b1c9f430a8e74994"
format_stamp: "Formatted at 2026-09-25 10:55:01 on dist-test-slave-nx7s"
server_key: "d7ad9f2377a10730f069a278ab647c0c"
server_key_iv: "88b4fc83663665f69343f2d74414449a"
server_key_version: "encryptionkey@0"
I20260925 10:55:01.410161 6257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:01.444146 6257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:01.448320 6257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:01.472692 6257 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.188:46577
I20260925 10:55:01.472725 6518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.188:46577 every 8 connection(s)
I20260925 10:55:01.473994 6257 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/master-2/data/info.pb
I20260925 10:55:01.480371 6519 sys_catalog.cc:263] Verifying existing consensus state
I20260925 10:55:01.484663 6519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: Bootstrap starting.
I20260925 10:55:01.503948 6519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 10:55:01.504494 6519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: Bootstrap complete.
I20260925 10:55:01.505618 6519 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } }
I20260925 10:55:01.505913 6519 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e6a2bc1541614ee3b1c9f430a8e74994, State: Initialized, Role: LEARNER
I20260925 10:55:01.506155 6519 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } }
I20260925 10:55:01.507246 6519 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:55:01.507045 6522 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } } }
I20260925 10:55:01.507597 6522 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [sys.catalog]: This master's current role is: LEARNER
I20260925 10:55:01.516916 6533 catalog_manager.cc:1259] Loaded cluster ID: 86416e3acf79441b9253cc5f1538d331
I20260925 10:55:01.517023 6533 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: loading cluster ID for follower catalog manager: success
I20260925 10:55:01.520447 6533 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: acquiring CA information for follower catalog manager: success
I20260925 10:55:01.522850 6533 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 10:55:01.743326 6456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:01.743628 6456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:01.743719 6456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:01.751096 6456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:01.751221 6456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:01.751343 6456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:55:01.761351 6456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:33393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.129.keytab
--principal=oryx/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:34941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=40919
--tserver_master_addrs=127.31.62.190:40377,127.31.62.189:36481,127.31.62.188:46577
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:01.762732 6456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:01.764115 6456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:01.778663 6538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:01.778625 6539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:01.778692 6541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:01.780169 6456 server_base.cc:1034] running on GCE node
I20260925 10:55:01.780965 6456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:01.782153 6456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:01.783409 6456 hybrid_clock.cc:648] HybridClock initialized: now 1790333701783359 us; error 52 us; skew 500 ppm
I20260925 10:55:01.783861 6456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:01 dist-test-slave-nx7s krb5kdc[5652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333701, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:01.788072 6456 init.cc:377] Logged in from keytab as oryx/127.31.62.129@KRBTEST.COM (short username oryx)
I20260925 10:55:01.789903 6456 webserver.cc:460] Webserver started at http://127.31.62.129:40919/ using document root <none> and password file <none>
I20260925 10:55:01.790650 6456 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:01.790895 6456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:01.797428 6456 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 10:55:01.800951 6548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:01.802654 6456 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 10:55:01.802870 6456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11"
format_stamp: "Formatted at 2026-09-25 10:54:58 on dist-test-slave-nx7s"
server_key: "626041e35044662ccd598c37e015879a"
server_key_iv: "878f9a60c4e53c7dc035b98a07a4a9b0"
server_key_version: "encryptionkey@0"
I20260925 10:55:01.804023 6456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:01.821382 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.810709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:46577 (local address 127.0.0.1:59134)
0925 10:55:01.811275 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:01.811296 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:01.811453 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:01.815780 (+ 4327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:01.815786 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:01.815822 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:01.815992 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.816003 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.817426 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.817432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.819272 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.819291 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.820590 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.820596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.820776 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.820813 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 10:55:01.820932 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 10:55:01.821074 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":310,"threads_started":1}
I20260925 10:55:01.821930 6555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.810798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59134 (local address 127.31.62.188:46577)
0925 10:55:01.815437 (+ 4639us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:01.815442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:01.815460 (+ 18us) server_negotiation.cc:407] Connection header received
0925 10:55:01.815522 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:01.815526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:01.815598 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:01.815706 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:01.816236 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.817264 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.819477 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.820410 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.820548 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.820738 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:01.820970 (+ 232us) server_negotiation.cc:299] Negotiation successful
0925 10:55:01.821158 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4525,"thread_start_us":4399,"threads_started":1}
I20260925 10:55:01.822788 6494 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260925 10:55:01.823325 6522 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44235815436447c3ab94d41adcbe92c5. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } } }
I20260925 10:55:01.823608 6522 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [sys.catalog]: This master's current role is: LEARNER
I20260925 10:55:01.827982 6456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:01.832549 6456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:01.832801 6456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:01.833847 6456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:01.835218 6456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:01.835315 6456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:01.835395 6456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:01.835448 6456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:01.875111 6456 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:34941
I20260925 10:55:01.875649 6663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:34941 every 8 connection(s)
I20260925 10:55:01.876983 6456 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:55:01.881909 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 6456
I20260925 10:55:01.882230 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 5892
Sep 25 10:55:01 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333701, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
I20260925 10:55:01.916778 6669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.880214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:45787 (local address 127.31.62.190:40377)
0925 10:55:01.893018 (+ 12804us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:01.893026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:01.893057 (+ 31us) server_negotiation.cc:407] Connection header received
0925 10:55:01.893142 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:01.893149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:01.893250 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:01.893417 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:01.897553 (+ 4136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.898766 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.901747 (+ 2981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.902394 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.903540 (+ 1146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.908326 (+ 4786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:01.908334 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:01.908338 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:01.908383 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:01.912018 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.913084 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.913090 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.913093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.913240 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.913813 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.913818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.913822 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.914152 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:01.914287 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:01.915590 (+ 1303us) server_negotiation.cc:299] Negotiation successful
0925 10:55:01.915829 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12618,"thread_start_us":13364,"threads_started":1}
Sep 25 10:55:01 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333701, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.189@KRBTEST.COM
I20260925 10:55:01.918285 6670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.881042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.129:45787)
0925 10:55:01.882122 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:01.882182 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:01.884679 (+ 2497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:01.895145 (+ 10466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:01.895163 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:01.896164 (+ 1001us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:01.897240 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.897277 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.899001 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.899008 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.901481 (+ 2473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.901504 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.902676 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.902684 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.903061 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.903984 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:01.904018 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:01.907988 (+ 3970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:01.912176 (+ 4188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.912186 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.912205 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.912928 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.913382 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.913387 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.913391 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.913660 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.914363 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:01.914371 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:01.915127 (+ 756us) client_negotiation.cc:769] Sending connection context
0925 10:55:01.915477 (+ 350us) client_negotiation.cc:240] Negotiation successful
0925 10:55:01.916085 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":828,"threads_started":1}
I20260925 10:55:01.920356 6665 heartbeater.cc:347] Connected to a master server at 127.31.62.190:40377
I20260925 10:55:01.920688 6665 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:01.921654 6665 heartbeater.cc:511] Master 127.31.62.190:40377 requested a full tablet report, sending...
I20260925 10:55:01.923899 5697 ts_manager.cc:194] Re-registered known tserver with Master: 1a0685c684ef4ad2a1d95ef3cf1f2a11 (127.31.62.129:34941)
Sep 25 10:55:01 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333701, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.188@KRBTEST.COM
I20260925 10:55:01.925037 5697 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:45787
I20260925 10:55:01.925148 6668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.880214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:57107 (local address 127.31.62.189:36481)
0925 10:55:01.880652 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:01.880656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:01.906402 (+ 25746us) server_negotiation.cc:407] Connection header received
0925 10:55:01.906990 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:01.906996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:01.907103 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:01.907177 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:01.909268 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.910562 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.912833 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.913537 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.913643 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.918296 (+ 4653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:01.918303 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:01.918308 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:01.918352 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:01.921662 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.922751 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.922769 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.922772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.922922 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.923343 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.923349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.923351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.923679 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:01.923853 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:01.924475 (+ 622us) server_negotiation.cc:299] Negotiation successful
0925 10:55:01.924704 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":241,"threads_started":1}
I20260925 10:55:01.926095 6674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.881819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:36481 (local address 127.31.62.129:57107)
0925 10:55:01.906270 (+ 24451us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:01.906295 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:01.906814 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:01.907446 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:01.907452 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:01.908287 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:01.909032 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.909049 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.910908 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.910914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.912652 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.912670 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.913947 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.913954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.914153 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.914803 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:01.914819 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:01.918062 (+ 3243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:01.921849 (+ 3787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.921854 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.921858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.922610 (+ 752us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.923059 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.923063 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.923067 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.923215 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.923937 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:01.923941 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:01.924095 (+ 154us) client_negotiation.cc:769] Sending connection context
0925 10:55:01.925389 (+ 1294us) client_negotiation.cc:240] Negotiation successful
0925 10:55:01.925692 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24278,"thread_start_us":32644,"threads_started":1}
Sep 25 10:55:01 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.62.130@KRBTEST.COM
I20260925 10:55:01.926959 6666 heartbeater.cc:347] Connected to a master server at 127.31.62.189:36481
I20260925 10:55:01.927115 6666 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:01.927772 6666 heartbeater.cc:511] Master 127.31.62.189:36481 requested a full tablet report, sending...
I20260925 10:55:01.929382 6297 ts_manager.cc:194] Registered new tserver with Master: 1a0685c684ef4ad2a1d95ef3cf1f2a11 (127.31.62.129:34941)
I20260925 10:55:01.931579 6555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.889150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:58121 (local address 127.31.62.188:46577)
0925 10:55:01.889347 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:01.889353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:01.914878 (+ 25525us) server_negotiation.cc:407] Connection header received
0925 10:55:01.915017 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:01.915022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:01.915102 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:01.915209 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:01.917496 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.918489 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.920289 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.920887 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.920976 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.925272 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:01.925279 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:01.925283 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:01.925326 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:01.928992 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.929765 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.929770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.929773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.929923 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.930378 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.930382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.930384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.930660 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:01.930779 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:01.931168 (+ 389us) server_negotiation.cc:299] Negotiation successful
0925 10:55:01.931351 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 10:55:01.931785 6671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.888533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:46577 (local address 127.31.62.129:58121)
0925 10:55:01.914763 (+ 26230us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:01.914781 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:01.914899 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:01.915269 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:01.915273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:01.915836 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:01.917303 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.917318 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.918657 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.918662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.920121 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:01.920145 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.921254 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.921260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:01.921436 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.921946 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:01.921970 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:01.925035 (+ 3065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:01.929159 (+ 4124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.929164 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.929168 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.929617 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.930069 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:01.930074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:01.930078 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:01.930249 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:01.930778 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:01.930783 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:01.930942 (+ 159us) client_negotiation.cc:769] Sending connection context
0925 10:55:01.931089 (+ 147us) client_negotiation.cc:240] Negotiation successful
0925 10:55:01.931415 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26092,"thread_start_us":17881,"threads_started":1}
I20260925 10:55:01.932773 6664 heartbeater.cc:347] Connected to a master server at 127.31.62.188:46577
I20260925 10:55:01.932921 6664 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:01.933256 6664 heartbeater.cc:511] Master 127.31.62.188:46577 requested a full tablet report, sending...
I20260925 10:55:01.934535 6673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:01.883616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.31.62.130:44351)
0925 10:55:01.911503 (+ 27887us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:01.911513 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:01.911546 (+ 33us) server_negotiation.cc:407] Connection header received
0925 10:55:01.911612 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:01.911621 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:01.911927 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:01.912088 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:01.916243 (+ 4155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.917940 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.919864 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:01.920726 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:01.920872 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:01.927908 (+ 7036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:01.927927 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:01.927947 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:01.927992 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:01.931491 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.932417 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.932426 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.932429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.932589 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:01.933001 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:01.933007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:01.933011 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:01.933361 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:01.933650 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:01.933959 (+ 309us) server_negotiation.cc:299] Negotiation successful
0925 10:55:01.934208 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27445,"thread_start_us":27520,"threads_started":1}
I20260925 10:55:01.934831 6484 ts_manager.cc:194] Registered new tserver with Master: 1a0685c684ef4ad2a1d95ef3cf1f2a11 (127.31.62.129:34941)
I20260925 10:55:01.935988 6015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:02.018983 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 5892
I20260925 10:55:02.031778 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:44351
--local_ip_for_outbound_sockets=127.31.62.130
--tserver_master_addrs=127.31.62.190:40377
--webserver_port=34745
--webserver_interface=127.31.62.130
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.130.keytab
--principal=oryx/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:33393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.62.190:40377,127.31.62.189:36481
--tserver_master_addrs=127.31.62.190:40377,127.31.62.189:36481,127.31.62.188:46577 with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:55:02.210124 6551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: NON_VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: true } } }
I20260925 10:55:02.210530 6551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:02.213152 6677 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5: attempting to promote NON_VOTER e6a2bc1541614ee3b1c9f430a8e74994 to VOTER
I20260925 10:55:02.213711 6677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } }
I20260925 10:55:02.215656 6307 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 FOLLOWER]: Refusing update from remote peer 44235815436447c3ab94d41adcbe92c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260925 10:55:02.216259 6494 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 1 LEARNER]: Refusing update from remote peer 44235815436447c3ab94d41adcbe92c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260925 10:55:02.216320 6677 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 10:55:02.216778 6678 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 10:55:02.219702 6551 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, e6a2bc1541614ee3b1c9f430a8e74994 (127.31.62.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } } }
I20260925 10:55:02.220340 6307 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, e6a2bc1541614ee3b1c9f430a8e74994 (127.31.62.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } } }
I20260925 10:55:02.225471 6494 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, e6a2bc1541614ee3b1c9f430a8e74994 (127.31.62.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } } }
I20260925 10:55:02.225943 6678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } } }
I20260925 10:55:02.226265 6678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:02.226506 6678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } } }
I20260925 10:55:02.226727 6678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:02.226917 6678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } } }
I20260925 10:55:02.227134 6678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:02.230299 6522 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } } }
I20260925 10:55:02.230597 6522 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 10:55:02.231997 6681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44235815436447c3ab94d41adcbe92c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } } }
I20260925 10:55:02.232388 6681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 10:55:02.380318 6676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:02.380614 6676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:02.380743 6676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:02.388276 6676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:02.388418 6676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:02.388566 6676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:55:02.398638 6676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:33393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.130.keytab
--principal=oryx/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:44351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=34745
--tserver_master_addrs=127.31.62.190:40377,127.31.62.189:36481,127.31.62.188:46577
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:02.400076 6676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:02.401443 6676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:02.414685 6686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:02.415108 6687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:02.415571 6689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:02.416013 6676 server_base.cc:1034] running on GCE node
I20260925 10:55:02.416731 6676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:02.417747 6676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:02.418959 6676 hybrid_clock.cc:648] HybridClock initialized: now 1790333702418895 us; error 60 us; skew 500 ppm
I20260925 10:55:02.419292 6676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:02 dist-test-slave-nx7s krb5kdc[5652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333702, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:02.422662 6676 init.cc:377] Logged in from keytab as oryx/127.31.62.130@KRBTEST.COM (short username oryx)
I20260925 10:55:02.424468 6676 webserver.cc:460] Webserver started at http://127.31.62.130:34745/ using document root <none> and password file <none>
I20260925 10:55:02.425391 6676 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:02.425554 6676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:02.431469 6676 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 10:55:02.434688 6696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:02.436254 6676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:55:02.436468 6676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "5017255eb8f34c7abd7e4ec3f63dc84f"
format_stamp: "Formatted at 2026-09-25 10:54:59 on dist-test-slave-nx7s"
server_key: "aaa7b4d4be8eb8b715be8b32ed503ad2"
server_key_iv: "168fcaa6b00d22daf69877f9688d2ea6"
server_key_version: "encryptionkey@0"
I20260925 10:55:02.437541 6676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:02.449800 6676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:02.454576 6676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:02.454867 6676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:02.455865 6676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:02.457268 6676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:02.457355 6676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:02.457449 6676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:02.457521 6676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:02.487784 6676 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:44351
I20260925 10:55:02.487860 6808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:44351 every 8 connection(s)
I20260925 10:55:02.489504 6676 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:55:02.491807 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 6676
I20260925 10:55:02.492076 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 6025
Sep 25 10:55:02 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333702, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.189@KRBTEST.COM
Sep 25 10:55:02 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333702, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.188@KRBTEST.COM
Sep 25 10:55:02 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333702, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
Sep 25 10:55:02 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.62.131@KRBTEST.COM
I20260925 10:55:02.530704 6819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:02.494409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:36643 (local address 127.31.62.188:46577)
0925 10:55:02.510004 (+ 15595us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:02.510010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:02.511931 (+ 1921us) server_negotiation.cc:407] Connection header received
0925 10:55:02.512092 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:02.512097 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:02.512183 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:02.512363 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:02.514064 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:02.515156 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:02.517143 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:02.517775 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:02.517891 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:02.523494 (+ 5603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:02.523502 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:02.523507 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:02.523549 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:02.526959 (+ 3410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:02.527951 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:02.527957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:02.527961 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:02.528099 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:02.528623 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:02.528628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:02.528631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:02.528955 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:02.529084 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:02.530066 (+ 982us) server_negotiation.cc:299] Negotiation successful
0925 10:55:02.530287 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15465,"thread_start_us":15315,"threads_started":1}
I20260925 10:55:02.531626 6817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:02.493937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:46577 (local address 127.31.62.130:36643)
0925 10:55:02.511823 (+ 17886us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:02.511841 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:02.511982 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:02.512535 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:02.512539 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:02.513184 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:02.513514 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:02.513525 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:02.515318 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:02.515323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:02.516983 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:02.517001 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:02.518127 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:02.518133 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:02.518376 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:02.519495 (+ 1119us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:02.519537 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:02.523166 (+ 3629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:02.527125 (+ 3959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:02.527135 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:02.527157 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:02.527785 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:02.528227 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:02.528232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:02.528236 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:02.528505 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:02.529079 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:02.529088 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:02.529562 (+ 474us) client_negotiation.cc:769] Sending connection context
0925 10:55:02.529898 (+ 336us) client_negotiation.cc:240] Negotiation successful
0925 10:55:02.530436 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17742,"mutex_wait_us":38,"spinlock_wait_cycles":26752,"thread_start_us":16746,"threads_started":1}
I20260925 10:55:02.531996 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:02.492578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:58153 (local address 127.31.62.189:36481)
0925 10:55:02.493107 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:02.493115 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:02.495125 (+ 2010us) server_negotiation.cc:407] Connection header received
0925 10:55:02.506434 (+ 11309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:02.506441 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:02.506546 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:02.506637 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:02.513102 (+ 6465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:02.514227 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:02.516940 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:02.517545 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:02.517653 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:02.523738 (+ 6085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:02.523745 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:02.523749 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:02.523803 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:02.527011 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:02.528426 (+ 1415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:02.528432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:02.528435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:02.528565 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:02.529447 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:02.529453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:02.529456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:02.529819 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:02.529957 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:02.531449 (+ 1492us) server_negotiation.cc:299] Negotiation successful
0925 10:55:02.531650 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":431,"threads_started":1}
I20260925 10:55:02.533214 6809 heartbeater.cc:347] Connected to a master server at 127.31.62.188:46577
I20260925 10:55:02.533313 6820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:02.495963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.130:53107)
0925 10:55:02.511882 (+ 15919us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:02.511902 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:02.512033 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:02.512365 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:02.512370 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:02.513022 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:02.513389 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:02.513403 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:02.516386 (+ 2983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:02.516391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:02.518031 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:02.518045 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:02.518818 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:02.518824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:02.518988 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:02.519531 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:02.519551 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:02.524109 (+ 4558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:02.528571 (+ 4462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:02.528575 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:02.528579 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:02.529068 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:02.529454 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:02.529459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:02.529462 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:02.529600 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:02.532364 (+ 2764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:02.532369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:02.532510 (+ 141us) client_negotiation.cc:769] Sending connection context
0925 10:55:02.532656 (+ 146us) client_negotiation.cc:240] Negotiation successful
0925 10:55:02.532937 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15787,"thread_start_us":15306,"threads_started":1}
I20260925 10:55:02.533560 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:02.493187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:36481 (local address 127.31.62.130:58153)
0925 10:55:02.494122 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:02.494173 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:02.503083 (+ 8910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:02.507328 (+ 4245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:02.507344 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:02.508279 (+ 935us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:02.509239 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:02.509268 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:02.514419 (+ 5151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:02.514424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:02.516725 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:02.516747 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:02.518007 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:02.518014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:02.518370 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:02.519245 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:02.519281 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:02.523509 (+ 4228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:02.527830 (+ 4321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:02.527835 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:02.527839 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:02.528311 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:02.529187 (+ 876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:02.529191 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:02.529194 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:02.529347 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:02.531000 (+ 1653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:02.531005 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:02.531171 (+ 166us) client_negotiation.cc:769] Sending connection context
0925 10:55:02.531349 (+ 178us) client_negotiation.cc:240] Negotiation successful
0925 10:55:02.531643 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":65,"spinlock_wait_cycles":21248,"thread_start_us":971,"threads_started":1}
I20260925 10:55:02.533710 6809 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:02.533951 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:02.492903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:53107 (local address 127.31.62.190:40377)
0925 10:55:02.511799 (+ 18896us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:02.511804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:02.512013 (+ 209us) server_negotiation.cc:407] Connection header received
0925 10:55:02.512154 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:02.512159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:02.512234 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:02.512328 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:02.514489 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:02.515562 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:02.518186 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:02.518625 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:02.518738 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:02.524832 (+ 6094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:02.524839 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:02.524844 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:02.524905 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:02.528447 (+ 3542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:02.529178 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:02.529183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:02.529186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:02.529332 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:02.529752 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:02.529766 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:02.529774 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:02.530133 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:02.530271 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:02.532809 (+ 2538us) server_negotiation.cc:299] Negotiation successful
0925 10:55:02.533070 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18778,"thread_start_us":18691,"threads_started":1}
I20260925 10:55:02.534251 6811 heartbeater.cc:347] Connected to a master server at 127.31.62.189:36481
I20260925 10:55:02.534420 6811 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:02.534564 6809 heartbeater.cc:511] Master 127.31.62.188:46577 requested a full tablet report, sending...
I20260925 10:55:02.534835 6811 heartbeater.cc:511] Master 127.31.62.189:36481 requested a full tablet report, sending...
I20260925 10:55:02.535872 6810 heartbeater.cc:347] Connected to a master server at 127.31.62.190:40377
I20260925 10:55:02.535977 6810 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:02.536195 6484 ts_manager.cc:194] Registered new tserver with Master: 5017255eb8f34c7abd7e4ec3f63dc84f (127.31.62.130:44351)
I20260925 10:55:02.536223 6297 ts_manager.cc:194] Registered new tserver with Master: 5017255eb8f34c7abd7e4ec3f63dc84f (127.31.62.130:44351)
I20260925 10:55:02.536340 6810 heartbeater.cc:511] Master 127.31.62.190:40377 requested a full tablet report, sending...
I20260925 10:55:02.537232 5697 ts_manager.cc:194] Re-registered known tserver with Master: 5017255eb8f34c7abd7e4ec3f63dc84f (127.31.62.130:44351)
I20260925 10:55:02.537923 6818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:02.494359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.31.62.131:34571)
0925 10:55:02.512243 (+ 17884us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:02.512252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:02.512281 (+ 29us) server_negotiation.cc:407] Connection header received
0925 10:55:02.512342 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:02.512349 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:02.512623 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:02.512785 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:02.514999 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:02.516215 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:02.518939 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:02.519877 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:02.520003 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:02.526087 (+ 6084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:02.526105 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:02.526124 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:02.526165 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:02.530632 (+ 4467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:02.532442 (+ 1810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:02.532452 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:02.532455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:02.532605 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:02.533120 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:02.533125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:02.533128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:02.535930 (+ 2802us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:02.536222 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:02.537433 (+ 1211us) server_negotiation.cc:299] Negotiation successful
0925 10:55:02.537640 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17705,"thread_start_us":16642,"threads_started":1}
I20260925 10:55:02.538035 5697 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:53107
I20260925 10:55:02.539294 6148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:02.622077 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 6025
I20260925 10:55:02.635124 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:34571
--local_ip_for_outbound_sockets=127.31.62.131
--tserver_master_addrs=127.31.62.190:40377
--webserver_port=39667
--webserver_interface=127.31.62.131
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.131.keytab
--principal=oryx/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:33393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.62.190:40377,127.31.62.189:36481
--tserver_master_addrs=127.31.62.190:40377,127.31.62.189:36481,127.31.62.188:46577 with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:55:02.929005 6665 heartbeater.cc:503] Master 127.31.62.190:40377 was elected leader, sending a full tablet report...
W20260925 10:55:02.963116 6822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:02.963438 6822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:02.963573 6822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:02.971040 6822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:02.971179 6822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:02.971333 6822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:55:02.981995 6822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:33393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.131.keytab
--principal=oryx/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:34571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=39667
--tserver_master_addrs=127.31.62.190:40377,127.31.62.189:36481,127.31.62.188:46577
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:02.983404 6822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:02.984723 6822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:02.998111 6827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:02.998162 6830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:02.998117 6828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:02.998744 6822 server_base.cc:1034] running on GCE node
I20260925 10:55:02.999444 6822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:03.000483 6822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:03.001718 6822 hybrid_clock.cc:648] HybridClock initialized: now 1790333703001649 us; error 63 us; skew 500 ppm
I20260925 10:55:03.002055 6822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:03 dist-test-slave-nx7s krb5kdc[5652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333702, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:03.005682 6822 init.cc:377] Logged in from keytab as oryx/127.31.62.131@KRBTEST.COM (short username oryx)
I20260925 10:55:03.007164 6822 webserver.cc:460] Webserver started at http://127.31.62.131:39667/ using document root <none> and password file <none>
I20260925 10:55:03.007893 6822 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:03.008026 6822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:03.013985 6822 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 10:55:03.016822 6837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:03.018353 6822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 10:55:03.018507 6822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "f727914630b045db8981c6190cd467dc"
format_stamp: "Formatted at 2026-09-25 10:54:59 on dist-test-slave-nx7s"
server_key: "20132df213316c986ce89e3bd953dd99"
server_key_iv: "c0daec8663ed5ad83c8433835c19c3cd"
server_key_version: "encryptionkey@0"
I20260925 10:55:03.019541 6822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:03.051124 6822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:03.055473 6822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:03.055754 6822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:03.056741 6822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:03.058151 6822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:03.058250 6822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:03.058357 6822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:03.058429 6822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:03.088042 6822 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:34571
I20260925 10:55:03.088109 6949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:34571 every 8 connection(s)
I20260925 10:55:03.089596 6822 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:55:03.094936 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 6822
Sep 25 10:55:03 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333702, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.189@KRBTEST.COM
Sep 25 10:55:03 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333702, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.188@KRBTEST.COM
Sep 25 10:55:03 dist-test-slave-nx7s krb5kdc[5652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333702, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
I20260925 10:55:03.141753 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.093029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:52345 (local address 127.31.62.189:36481)
0925 10:55:03.093647 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:03.093654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:03.116124 (+ 22470us) server_negotiation.cc:407] Connection header received
0925 10:55:03.116413 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:03.116420 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:03.116517 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:03.116602 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:03.122311 (+ 5709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.123374 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.126286 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.126976 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.127076 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.132768 (+ 5692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:03.132778 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:03.132783 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:03.132823 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:03.136125 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:03.137026 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:03.137031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:03.137035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:03.137207 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:03.137925 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:03.137930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:03.137934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:03.138270 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:03.138392 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:03.141040 (+ 2648us) server_negotiation.cc:299] Negotiation successful
0925 10:55:03.141356 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":383,"threads_started":1}
I20260925 10:55:03.144506 6958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.093275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:36481 (local address 127.31.62.131:52345)
0925 10:55:03.115651 (+ 22376us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:03.115984 (+ 333us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:03.116262 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:03.117295 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:03.117311 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:03.118393 (+ 1082us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:03.119388 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.119418 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.123547 (+ 4129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.123552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.126068 (+ 2516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.126086 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.127469 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.127476 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.127806 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.128586 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:03.128617 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:03.131996 (+ 3379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:03.136275 (+ 4279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:03.136285 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:03.136304 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:03.136893 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:03.137360 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:03.137365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:03.137369 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:03.137604 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:03.138480 (+ 876us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:03.138489 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:03.138924 (+ 435us) client_negotiation.cc:769] Sending connection context
0925 10:55:03.142031 (+ 3107us) client_negotiation.cc:240] Negotiation successful
0925 10:55:03.142576 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22162,"mutex_wait_us":25,"spinlock_wait_cycles":50048,"thread_start_us":22269,"threads_started":1}
I20260925 10:55:03.146379 6952 heartbeater.cc:347] Connected to a master server at 127.31.62.189:36481
I20260925 10:55:03.146695 6952 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:03.147542 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.092681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:51825 (local address 127.31.62.188:46577)
0925 10:55:03.121522 (+ 28841us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:03.121529 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:03.121549 (+ 20us) server_negotiation.cc:407] Connection header received
0925 10:55:03.121612 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:03.121618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:03.121700 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:03.124280 (+ 2580us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:03.124497 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.125864 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.127879 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.128592 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.129513 (+ 921us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.132504 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:03.132511 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:03.132525 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:03.132565 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:03.138635 (+ 6070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:03.144837 (+ 6202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:03.144842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:03.144866 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:03.145011 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:03.145449 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:03.145453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:03.145456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:03.145816 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:03.146853 (+ 1037us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:03.146989 (+ 136us) server_negotiation.cc:299] Negotiation successful
0925 10:55:03.147212 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28693,"thread_start_us":28570,"threads_started":1}
I20260925 10:55:03.147567 6952 heartbeater.cc:511] Master 127.31.62.189:36481 requested a full tablet report, sending...
I20260925 10:55:03.148502 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.093275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:46577 (local address 127.31.62.131:51825)
0925 10:55:03.101172 (+ 7897us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:03.101252 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:03.104068 (+ 2816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:03.121895 (+ 17827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:03.121902 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:03.123372 (+ 1470us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:03.123765 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.123783 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.126101 (+ 2318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.126109 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.127656 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.127671 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.128825 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.128837 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.129124 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.129761 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:03.129789 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:03.132255 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:03.144069 (+ 11814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:03.144073 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:03.144076 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:03.144531 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:03.145142 (+ 611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:03.145146 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:03.145149 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:03.145309 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:03.145935 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:03.145939 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:03.146091 (+ 152us) client_negotiation.cc:769] Sending connection context
0925 10:55:03.146214 (+ 123us) client_negotiation.cc:240] Negotiation successful
0925 10:55:03.146494 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":884,"thread_start_us":29001,"threads_started":1}
I20260925 10:55:03.149461 6297 ts_manager.cc:194] Registered new tserver with Master: f727914630b045db8981c6190cd467dc (127.31.62.131:34571)
I20260925 10:55:03.149456 6950 heartbeater.cc:347] Connected to a master server at 127.31.62.188:46577
I20260925 10:55:03.149618 6950 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:03.150188 6950 heartbeater.cc:511] Master 127.31.62.188:46577 requested a full tablet report, sending...
I20260925 10:55:03.151389 6484 ts_manager.cc:194] Registered new tserver with Master: f727914630b045db8981c6190cd467dc (127.31.62.131:34571)
I20260925 10:55:03.153898 6954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.092473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:42811 (local address 127.31.62.190:40377)
0925 10:55:03.122051 (+ 29578us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:03.122058 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:03.122925 (+ 867us) server_negotiation.cc:407] Connection header received
0925 10:55:03.123135 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:03.123141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:03.123226 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:03.123324 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:03.129378 (+ 6054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.130695 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.132724 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.133758 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.133859 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.139481 (+ 5622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:03.139488 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:03.139491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:03.139529 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:03.142880 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:03.151500 (+ 8620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:03.151505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:03.151508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:03.151689 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:03.152123 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:03.152129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:03.152132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:03.152453 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:03.152600 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:03.153184 (+ 584us) server_negotiation.cc:299] Negotiation successful
0925 10:55:03.153445 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29439,"thread_start_us":29587,"threads_started":1}
I20260925 10:55:03.154938 6957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.093287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.131:42811)
0925 10:55:03.122793 (+ 29506us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:03.122817 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:03.122960 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:03.123488 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:03.123492 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:03.128788 (+ 5296us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:03.129193 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.129205 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.130934 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.130939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.132508 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.132524 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.134115 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.134121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.134286 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.134946 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:03.134968 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:03.139289 (+ 4321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:03.143045 (+ 3756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:03.143049 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:03.143053 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:03.143927 (+ 874us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:03.151832 (+ 7905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:03.151837 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:03.151840 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:03.152009 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:03.152616 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:03.152622 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:03.152819 (+ 197us) client_negotiation.cc:769] Sending connection context
0925 10:55:03.154156 (+ 1337us) client_negotiation.cc:240] Negotiation successful
0925 10:55:03.154478 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29346,"mutex_wait_us":365,"thread_start_us":350,"threads_started":1}
I20260925 10:55:03.155880 6951 heartbeater.cc:347] Connected to a master server at 127.31.62.190:40377
I20260925 10:55:03.156030 6951 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:03.157052 6951 heartbeater.cc:511] Master 127.31.62.190:40377 requested a full tablet report, sending...
I20260925 10:55:03.158118 5697 ts_manager.cc:194] Re-registered known tserver with Master: f727914630b045db8981c6190cd467dc (127.31.62.131:34571)
I20260925 10:55:03.159162 5697 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:42811
I20260925 10:55:03.166329 6965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.124155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35324 (local address 127.31.62.190:40377)
0925 10:55:03.134495 (+ 10340us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:03.134501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:03.134519 (+ 18us) server_negotiation.cc:407] Connection header received
0925 10:55:03.134566 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:03.134571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:03.134643 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:03.137712 (+ 3069us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:03.150200 (+ 12488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.151315 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.154778 (+ 3463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.155798 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.155914 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.159591 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:03.159598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:03.159603 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:03.159637 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:03.162621 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:03.163897 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:03.163901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:03.163903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:03.164450 (+ 547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:03.164900 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:03.164905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:03.164908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:03.165167 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:03.165379 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:03.165710 (+ 331us) server_negotiation.cc:299] Negotiation successful
0925 10:55:03.165909 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9372,"thread_start_us":4305,"threads_started":1}
I20260925 10:55:03.171602 6966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.127293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52092 (local address 127.31.62.189:36481)
0925 10:55:03.130996 (+ 3703us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:03.131002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:03.152721 (+ 21719us) server_negotiation.cc:407] Connection header received
0925 10:55:03.152911 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:03.152916 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:03.152992 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:03.153086 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:03.156025 (+ 2939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.157598 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.161166 (+ 3568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.161884 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.161994 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.165456 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:03.165464 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:03.165468 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:03.165511 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:03.168592 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:03.169442 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:03.169447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:03.169451 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:03.169679 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:03.170028 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:03.170033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:03.170036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:03.170379 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:03.170623 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:03.170955 (+ 332us) server_negotiation.cc:299] Negotiation successful
0925 10:55:03.171228 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3577,"thread_start_us":3774,"threads_started":1}
I20260925 10:55:03.173103 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.132482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59140 (local address 127.31.62.188:46577)
0925 10:55:03.147970 (+ 15488us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:03.147976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:03.157193 (+ 9217us) server_negotiation.cc:407] Connection header received
0925 10:55:03.157255 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:03.157261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:03.157348 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:03.157444 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:03.159323 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.161001 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.163191 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.163670 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.163760 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.166874 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:03.166893 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:03.166897 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:03.166937 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:03.170082 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:03.170802 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:03.170807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:03.170811 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:03.170957 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:03.171372 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:03.171378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:03.171382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:03.171726 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:03.171948 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:03.172497 (+ 549us) server_negotiation.cc:299] Negotiation successful
0925 10:55:03.172723 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15385,"thread_start_us":23162,"threads_started":1}
I20260925 10:55:03.170282 5697 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:55:03.178552 5697 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:55:03.222191 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.205791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44474 (local address 127.31.62.129:34941)
0925 10:55:03.206774 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:03.206795 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:03.213934 (+ 7139us) server_negotiation.cc:407] Connection header received
0925 10:55:03.214330 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:03.214338 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:03.214646 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:03.214887 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:03.215758 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.217268 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.219774 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.221107 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.221239 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.221466 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:03.221707 (+ 241us) server_negotiation.cc:299] Negotiation successful
0925 10:55:03.221871 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":278,"threads_started":1}
I20260925 10:55:03.222401 6982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.205530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34941 (local address 127.0.0.1:44474)
0925 10:55:03.213770 (+ 8240us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:03.213793 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:03.214062 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:03.215180 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:03.215185 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:03.215217 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:03.215564 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.215575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.217424 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.217431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.219610 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.219624 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.221323 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.221329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.221514 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.221551 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 10:55:03.221652 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 10:55:03.221809 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8104,"mutex_wait_us":1,"thread_start_us":7923,"threads_started":1}
I20260925 10:55:03.223031 6981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.204062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47650 (local address 127.31.62.130:44351)
0925 10:55:03.213346 (+ 9284us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:03.213358 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:03.213400 (+ 42us) server_negotiation.cc:407] Connection header received
0925 10:55:03.213524 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:03.213534 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:03.213891 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:03.215267 (+ 1376us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:03.215466 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.216952 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.219497 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.221070 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.221230 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.221459 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:03.222513 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0925 10:55:03.222706 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9112,"thread_start_us":9006,"threads_started":1}
I20260925 10:55:03.223577 6980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.203868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:44351 (local address 127.0.0.1:47650)
0925 10:55:03.204422 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:03.204442 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:03.204561 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:03.214354 (+ 9793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:03.214360 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:03.214391 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:03.214819 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.214834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.217146 (+ 2312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.217152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.219287 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.219303 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.221621 (+ 2318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.221628 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.221885 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.221933 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 10:55:03.222049 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 10:55:03.222214 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":56,"thread_start_us":305,"threads_started":1}
I20260925 10:55:03.228387 6978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.202921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:34571 (local address 127.0.0.1:37846)
0925 10:55:03.203632 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:03.203652 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:03.203783 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:03.218645 (+ 14862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:03.218652 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:03.218690 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:03.219223 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.219237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.221802 (+ 2565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.221807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.224106 (+ 2299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.224124 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.227297 (+ 3173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.227305 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.227599 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.227645 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 10:55:03.227737 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 10:55:03.227900 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":18,"thread_start_us":421,"threads_started":1}
I20260925 10:55:03.228289 6744 tablet_service.cc:1467] Processing CreateTablet for tablet eed11e666ba642cc89187713a19f175f (DEFAULT_TABLE table=test-table [id=a66618cebabb4557bc779204c5b1d1e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:03.229373 6599 tablet_service.cc:1467] Processing CreateTablet for tablet eed11e666ba642cc89187713a19f175f (DEFAULT_TABLE table=test-table [id=a66618cebabb4557bc779204c5b1d1e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:03.230505 6744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eed11e666ba642cc89187713a19f175f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:03.230935 6599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eed11e666ba642cc89187713a19f175f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:03.231124 6979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.203630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37846 (local address 127.31.62.131:34571)
0925 10:55:03.217928 (+ 14298us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:03.217937 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:03.218000 (+ 63us) server_negotiation.cc:407] Connection header received
0925 10:55:03.218105 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:03.218114 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:03.218411 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:03.218634 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:03.219509 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.221627 (+ 2118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.225342 (+ 3715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.226967 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.228904 (+ 1937us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.229508 (+ 604us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:03.229621 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 10:55:03.229831 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14107,"thread_start_us":13402,"threads_started":1}
I20260925 10:55:03.235436 6885 tablet_service.cc:1467] Processing CreateTablet for tablet eed11e666ba642cc89187713a19f175f (DEFAULT_TABLE table=test-table [id=a66618cebabb4557bc779204c5b1d1e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:03.237191 6885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eed11e666ba642cc89187713a19f175f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:03.247751 6984 tablet_bootstrap.cc:492] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f: Bootstrap starting.
I20260925 10:55:03.251031 6984 tablet_bootstrap.cc:654] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:03.252754 6984 log.cc:824] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:03.253525 6985 tablet_bootstrap.cc:492] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Bootstrap starting.
I20260925 10:55:03.257191 6987 tablet_bootstrap.cc:492] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc: Bootstrap starting.
I20260925 10:55:03.258150 6985 tablet_bootstrap.cc:654] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:03.259742 6985 log.cc:824] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:03.261193 6987 tablet_bootstrap.cc:654] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:03.262182 6987 log.cc:824] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:03.262892 6984 tablet_bootstrap.cc:492] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f: No bootstrap required, opened a new log
I20260925 10:55:03.262892 6985 tablet_bootstrap.cc:492] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11: No bootstrap required, opened a new log
I20260925 10:55:03.263195 6984 ts_tablet_manager.cc:1397] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.004s
I20260925 10:55:03.263195 6985 ts_tablet_manager.cc:1397] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260925 10:55:03.264899 6987 tablet_bootstrap.cc:492] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc: No bootstrap required, opened a new log
I20260925 10:55:03.265169 6987 ts_tablet_manager.cc:1397] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260925 10:55:03.273658 6987 raft_consensus.cc:348] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } } peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } }
I20260925 10:55:03.274194 6987 raft_consensus.cc:374] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:03.274375 6987 raft_consensus.cc:729] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f727914630b045db8981c6190cd467dc, State: Initialized, Role: FOLLOWER
I20260925 10:55:03.275168 6987 consensus_queue.cc:260] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } } peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } }
I20260925 10:55:03.275457 6984 raft_consensus.cc:348] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } } peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } }
I20260925 10:55:03.275557 6985 raft_consensus.cc:348] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } } peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } }
I20260925 10:55:03.276115 6984 raft_consensus.cc:374] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:03.276115 6985 raft_consensus.cc:374] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:03.276291 6985 raft_consensus.cc:729] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a0685c684ef4ad2a1d95ef3cf1f2a11, State: Initialized, Role: FOLLOWER
I20260925 10:55:03.276292 6984 raft_consensus.cc:729] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5017255eb8f34c7abd7e4ec3f63dc84f, State: Initialized, Role: FOLLOWER
I20260925 10:55:03.277173 6984 consensus_queue.cc:260] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } } peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } }
I20260925 10:55:03.277616 6951 heartbeater.cc:503] Master 127.31.62.190:40377 was elected leader, sending a full tablet report...
I20260925 10:55:03.278088 6987 ts_tablet_manager.cc:1428] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260925 10:55:03.281736 6985 consensus_queue.cc:260] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } } peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } }
I20260925 10:55:03.286396 6810 heartbeater.cc:503] Master 127.31.62.190:40377 was elected leader, sending a full tablet report...
I20260925 10:55:03.287281 6985 ts_tablet_manager.cc:1428] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Time spent starting tablet: real 0.024s user 0.020s sys 0.000s
I20260925 10:55:03.287756 6984 ts_tablet_manager.cc:1428] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f: Time spent starting tablet: real 0.024s user 0.013s sys 0.012s
I20260925 10:55:03.335804 6992 raft_consensus.cc:482] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:55:03.336447 6992 raft_consensus.cc:504] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } } peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } }
I20260925 10:55:03.338855 6992 leader_election.cc:288] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f727914630b045db8981c6190cd467dc (127.31.62.131:34571), 5017255eb8f34c7abd7e4ec3f63dc84f (127.31.62.130:44351)
W20260925 10:55:03.342567 6953 tablet.cc:2369] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:55:03.344379 6991 raft_consensus.cc:482] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:55:03.344704 6991 raft_consensus.cc:504] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } } peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } }
I20260925 10:55:03.346516 6991 leader_election.cc:288] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f727914630b045db8981c6190cd467dc (127.31.62.131:34571), 1a0685c684ef4ad2a1d95ef3cf1f2a11 (127.31.62.129:34941)
I20260925 10:55:03.350718 6979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.339062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:41165 (local address 127.31.62.131:34571)
0925 10:55:03.340268 (+ 1206us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:03.340275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:03.341221 (+ 946us) server_negotiation.cc:407] Connection header received
0925 10:55:03.341467 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:03.341472 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:03.341538 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:03.341629 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:03.342937 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.344144 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.348229 (+ 4085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.349474 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.349585 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.349658 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:03.350324 (+ 666us) server_negotiation.cc:299] Negotiation successful
0925 10:55:03.350459 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 10:55:03.350708 6981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.339448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:53209 (local address 127.31.62.130:44351)
0925 10:55:03.339729 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:03.339735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:03.341190 (+ 1455us) server_negotiation.cc:407] Connection header received
0925 10:55:03.341435 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:03.341441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:03.341519 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:03.341633 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:03.342880 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.344296 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.348058 (+ 3762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.349387 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.349518 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.349591 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:03.350321 (+ 730us) server_negotiation.cc:299] Negotiation successful
0925 10:55:03.350459 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 10:55:03.351415 6993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.339062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:44351 (local address 127.31.62.129:53209)
0925 10:55:03.341025 (+ 1963us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:03.341049 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:03.341287 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:03.341853 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:03.341857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:03.341880 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:03.342636 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.342651 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.344511 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.344516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.347886 (+ 3370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.347899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.349838 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.349849 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.350070 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.350166 (+ 96us) client_negotiation.cc:769] Sending connection context
0925 10:55:03.350896 (+ 730us) client_negotiation.cc:240] Negotiation successful
0925 10:55:03.351046 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1743,"mutex_wait_us":124,"thread_start_us":1312,"threads_started":1}
I20260925 10:55:03.352396 6764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed11e666ba642cc89187713a19f175f" candidate_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" is_pre_election: true
I20260925 10:55:03.353019 6764 raft_consensus.cc:2454] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a0685c684ef4ad2a1d95ef3cf1f2a11 in term 0.
I20260925 10:55:03.353812 6549 leader_election.cc:302] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a0685c684ef4ad2a1d95ef3cf1f2a11, 5017255eb8f34c7abd7e4ec3f63dc84f; no voters:
I20260925 10:55:03.355723 6994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.339159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:34571 (local address 127.31.62.129:41165)
0925 10:55:03.341117 (+ 1958us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:03.341131 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:03.341329 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:03.341682 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:03.341686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:03.341709 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:03.342501 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.342518 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.344335 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.344340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.347693 (+ 3353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.347712 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.349863 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.349869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.350073 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.350162 (+ 89us) client_negotiation.cc:769] Sending connection context
0925 10:55:03.355122 (+ 4960us) client_negotiation.cc:240] Negotiation successful
0925 10:55:03.355297 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1751,"mutex_wait_us":58,"spinlock_wait_cycles":85376,"thread_start_us":1399,"threads_started":1}
I20260925 10:55:03.355954 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.347985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:33555 (local address 127.31.62.129:34941)
0925 10:55:03.348167 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:03.348173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:03.348230 (+ 57us) server_negotiation.cc:407] Connection header received
0925 10:55:03.348478 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:03.348484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:03.348561 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:03.348626 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:03.349488 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.350710 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.353415 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.354721 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.354860 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.354936 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:03.355519 (+ 583us) server_negotiation.cc:299] Negotiation successful
0925 10:55:03.355668 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 10:55:03.356660 6997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.347659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34941 (local address 127.31.62.130:33555)
0925 10:55:03.348136 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:03.348156 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:03.348328 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:03.348819 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:03.348824 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:03.348891 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:03.349297 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.349308 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.350893 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.350897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.353220 (+ 2323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.353235 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.355136 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.355142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.355321 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.355357 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 10:55:03.356118 (+ 761us) client_negotiation.cc:240] Negotiation successful
0925 10:55:03.356282 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":268,"threads_started":1}
I20260925 10:55:03.357223 6995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.346872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:34571 (local address 127.31.62.130:56089)
0925 10:55:03.347493 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:03.347527 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:03.347646 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:03.351418 (+ 3772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:03.351423 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:03.351447 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:03.351995 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.352008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.353450 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.353454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.354972 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:03.354984 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.356506 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.356512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:03.356664 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.356687 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 10:55:03.356770 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 10:55:03.356931 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":1230,"threads_started":1}
I20260925 10:55:03.357558 6979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:03.347271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:56089 (local address 127.31.62.131:34571)
0925 10:55:03.351018 (+ 3747us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:03.351024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:03.351040 (+ 16us) server_negotiation.cc:407] Connection header received
0925 10:55:03.351096 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:03.351101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:03.351174 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:03.351272 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:03.352164 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.353297 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.355131 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:03.356300 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:03.356441 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:03.356511 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:03.357244 (+ 733us) server_negotiation.cc:299] Negotiation successful
0925 10:55:03.357373 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3645,"threads_started":1}
I20260925 10:55:03.357791 6619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed11e666ba642cc89187713a19f175f" candidate_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" is_pre_election: true
I20260925 10:55:03.358237 6619 raft_consensus.cc:2454] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5017255eb8f34c7abd7e4ec3f63dc84f in term 0.
I20260925 10:55:03.358498 6992 raft_consensus.cc:2781] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:55:03.358655 6992 raft_consensus.cc:482] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:55:03.358734 6992 raft_consensus.cc:3037] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:03.359074 6698 leader_election.cc:302] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a0685c684ef4ad2a1d95ef3cf1f2a11, 5017255eb8f34c7abd7e4ec3f63dc84f; no voters:
I20260925 10:55:03.359638 6991 raft_consensus.cc:2781] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:55:03.359781 6991 raft_consensus.cc:482] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:55:03.359859 6991 raft_consensus.cc:3037] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:03.361502 6992 raft_consensus.cc:504] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } } peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } }
I20260925 10:55:03.362880 6991 raft_consensus.cc:504] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } } peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } }
I20260925 10:55:03.362970 6764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed11e666ba642cc89187713a19f175f" candidate_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f"
I20260925 10:55:03.363368 6992 leader_election.cc:288] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [CANDIDATE]: Term 1 election: Requested vote from peers f727914630b045db8981c6190cd467dc (127.31.62.131:34571), 5017255eb8f34c7abd7e4ec3f63dc84f (127.31.62.130:44351)
I20260925 10:55:03.363646 6991 leader_election.cc:288] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [CANDIDATE]: Term 1 election: Requested vote from peers f727914630b045db8981c6190cd467dc (127.31.62.131:34571), 1a0685c684ef4ad2a1d95ef3cf1f2a11 (127.31.62.129:34941)
I20260925 10:55:03.365357 6764 raft_consensus.cc:2379] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1a0685c684ef4ad2a1d95ef3cf1f2a11 in current term 1: Already voted for candidate 5017255eb8f34c7abd7e4ec3f63dc84f in this term.
I20260925 10:55:03.365567 6619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed11e666ba642cc89187713a19f175f" candidate_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11"
I20260925 10:55:03.365902 6619 raft_consensus.cc:2379] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5017255eb8f34c7abd7e4ec3f63dc84f in current term 1: Already voted for candidate 1a0685c684ef4ad2a1d95ef3cf1f2a11 in this term.
I20260925 10:55:03.370687 6903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed11e666ba642cc89187713a19f175f" candidate_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f727914630b045db8981c6190cd467dc" is_pre_election: true
I20260925 10:55:03.370659 6904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed11e666ba642cc89187713a19f175f" candidate_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f727914630b045db8981c6190cd467dc"
I20260925 10:55:03.371016 6905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed11e666ba642cc89187713a19f175f" candidate_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f727914630b045db8981c6190cd467dc" is_pre_election: true
I20260925 10:55:03.371193 6902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed11e666ba642cc89187713a19f175f" candidate_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f727914630b045db8981c6190cd467dc"
I20260925 10:55:03.371470 6903 raft_consensus.cc:2454] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a0685c684ef4ad2a1d95ef3cf1f2a11 in term 0.
I20260925 10:55:03.372195 6699 leader_election.cc:302] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5017255eb8f34c7abd7e4ec3f63dc84f; no voters: 1a0685c684ef4ad2a1d95ef3cf1f2a11, f727914630b045db8981c6190cd467dc
I20260925 10:55:03.372464 6552 leader_election.cc:302] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a0685c684ef4ad2a1d95ef3cf1f2a11; no voters: 5017255eb8f34c7abd7e4ec3f63dc84f, f727914630b045db8981c6190cd467dc
I20260925 10:55:03.372516 6991 raft_consensus.cc:2726] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 10:55:03.372886 6992 raft_consensus.cc:2726] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260925 10:55:03.387756 6667 tablet.cc:2369] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:55:03.492275 6812 tablet.cc:2369] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:55:05.240097 6999 raft_consensus.cc:482] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:55:05.240307 6999 raft_consensus.cc:504] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } } peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } }
I20260925 10:55:05.241971 6999 leader_election.cc:288] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5017255eb8f34c7abd7e4ec3f63dc84f (127.31.62.130:44351), 1a0685c684ef4ad2a1d95ef3cf1f2a11 (127.31.62.129:34941)
I20260925 10:55:05.260586 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:05.242392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:44371 (local address 127.31.62.130:44351)
0925 10:55:05.255576 (+ 13184us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:05.255580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:05.255596 (+ 16us) server_negotiation.cc:407] Connection header received
0925 10:55:05.255658 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:05.255663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:05.255735 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:05.255841 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:05.256544 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:05.257331 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:05.258892 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:05.259625 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:05.259774 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:05.259854 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:05.260171 (+ 317us) server_negotiation.cc:299] Negotiation successful
0925 10:55:05.260312 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13068,"thread_start_us":12978,"threads_started":1}
I20260925 10:55:05.260586 7003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:05.242482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:47453 (local address 127.31.62.129:34941)
0925 10:55:05.254691 (+ 12209us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:05.254699 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:05.255511 (+ 812us) server_negotiation.cc:407] Connection header received
0925 10:55:05.255665 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:05.255670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:05.255735 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:05.255849 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:05.256389 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:05.257137 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:05.258858 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:05.259645 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:05.259792 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:05.259864 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:05.260196 (+ 332us) server_negotiation.cc:299] Negotiation successful
0925 10:55:05.260334 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12047,"thread_start_us":11921,"threads_started":1}
I20260925 10:55:05.260704 7000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:05.242173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:44351 (local address 127.31.62.131:44371)
0925 10:55:05.255024 (+ 12851us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:05.255050 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:05.255241 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:05.256034 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:05.256042 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:05.256063 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:05.256389 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:05.256399 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:05.257475 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:05.257479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:05.258733 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:05.258745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:05.259798 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:05.259808 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:05.260010 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:05.260046 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 10:55:05.260116 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 10:55:05.260254 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12669,"mutex_wait_us":26,"thread_start_us":12556,"threads_started":1}
I20260925 10:55:05.261149 7001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:05.242277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34941 (local address 127.31.62.131:47453)
0925 10:55:05.255377 (+ 13100us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:05.255397 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:05.255541 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:05.255887 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:05.255891 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:05.255914 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:05.256130 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:05.256140 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:05.257271 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:05.257276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:05.258708 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:05.258719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:05.259821 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:05.259826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:05.260048 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:05.260072 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 10:55:05.260156 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 10:55:05.260286 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12967,"mutex_wait_us":45,"thread_start_us":12841,"threads_started":1}
I20260925 10:55:05.261394 6764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed11e666ba642cc89187713a19f175f" candidate_uuid: "f727914630b045db8981c6190cd467dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" is_pre_election: true
I20260925 10:55:05.261657 6764 raft_consensus.cc:2379] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f727914630b045db8981c6190cd467dc in current term 1: Already voted for candidate 5017255eb8f34c7abd7e4ec3f63dc84f in this term.
I20260925 10:55:05.261750 6619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed11e666ba642cc89187713a19f175f" candidate_uuid: "f727914630b045db8981c6190cd467dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" is_pre_election: true
I20260925 10:55:05.262001 6619 raft_consensus.cc:2379] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f727914630b045db8981c6190cd467dc in current term 1: Already voted for candidate 1a0685c684ef4ad2a1d95ef3cf1f2a11 in this term.
I20260925 10:55:05.262459 6839 leader_election.cc:302] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f727914630b045db8981c6190cd467dc; no voters: 1a0685c684ef4ad2a1d95ef3cf1f2a11, 5017255eb8f34c7abd7e4ec3f63dc84f
I20260925 10:55:05.262876 6999 raft_consensus.cc:3037] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:05.265195 6999 raft_consensus.cc:2726] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:55:05.411648 7004 raft_consensus.cc:482] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:55:05.411826 7004 raft_consensus.cc:504] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } } peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } }
I20260925 10:55:05.412592 7004 leader_election.cc:288] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f727914630b045db8981c6190cd467dc (127.31.62.131:34571), 1a0685c684ef4ad2a1d95ef3cf1f2a11 (127.31.62.129:34941)
I20260925 10:55:05.413254 6619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed11e666ba642cc89187713a19f175f" candidate_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" is_pre_election: true
I20260925 10:55:05.413331 6905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed11e666ba642cc89187713a19f175f" candidate_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f727914630b045db8981c6190cd467dc" is_pre_election: true
I20260925 10:55:05.413487 6619 raft_consensus.cc:2454] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5017255eb8f34c7abd7e4ec3f63dc84f in term 1.
I20260925 10:55:05.413599 6905 raft_consensus.cc:2454] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5017255eb8f34c7abd7e4ec3f63dc84f in term 1.
I20260925 10:55:05.413944 6698 leader_election.cc:302] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a0685c684ef4ad2a1d95ef3cf1f2a11, 5017255eb8f34c7abd7e4ec3f63dc84f; no voters:
I20260925 10:55:05.414258 7004 raft_consensus.cc:2781] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 10:55:05.414366 7004 raft_consensus.cc:482] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:55:05.414448 7004 raft_consensus.cc:3037] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 1 FOLLOWER]: Advancing to term 2
I20260925 10:55:05.416482 7004 raft_consensus.cc:504] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } } peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } }
I20260925 10:55:05.417063 7004 leader_election.cc:288] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [CANDIDATE]: Term 2 election: Requested vote from peers f727914630b045db8981c6190cd467dc (127.31.62.131:34571), 1a0685c684ef4ad2a1d95ef3cf1f2a11 (127.31.62.129:34941)
I20260925 10:55:05.417662 6619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed11e666ba642cc89187713a19f175f" candidate_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11"
I20260925 10:55:05.417677 6905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed11e666ba642cc89187713a19f175f" candidate_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f727914630b045db8981c6190cd467dc"
I20260925 10:55:05.417862 6619 raft_consensus.cc:3037] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 1 FOLLOWER]: Advancing to term 2
I20260925 10:55:05.417897 6905 raft_consensus.cc:3037] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [term 1 FOLLOWER]: Advancing to term 2
I20260925 10:55:05.419919 6619 raft_consensus.cc:2454] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5017255eb8f34c7abd7e4ec3f63dc84f in term 2.
I20260925 10:55:05.420022 6905 raft_consensus.cc:2454] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5017255eb8f34c7abd7e4ec3f63dc84f in term 2.
I20260925 10:55:05.420430 6698 leader_election.cc:302] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a0685c684ef4ad2a1d95ef3cf1f2a11, 5017255eb8f34c7abd7e4ec3f63dc84f; no voters:
I20260925 10:55:05.420768 7004 raft_consensus.cc:2781] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 2 FOLLOWER]: Leader election won for term 2
I20260925 10:55:05.421165 7004 raft_consensus.cc:686] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 2 LEADER]: Becoming Leader. State: Replica: 5017255eb8f34c7abd7e4ec3f63dc84f, State: Running, Role: LEADER
I20260925 10:55:05.421705 7004 consensus_queue.cc:237] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } } peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } }
I20260925 10:55:05.425966 5697 catalog_manager.cc:5495] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f reported cstate change: term changed from 0 to 2, leader changed from <none> to 5017255eb8f34c7abd7e4ec3f63dc84f (127.31.62.130). New cstate: current_term: 2 leader_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } health_report { overall_health: UNKNOWN } } }
I20260925 10:55:05.843910 7019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:05.838347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53318 (local address 127.31.62.130:44351)
0925 10:55:05.838747 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:05.838786 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:05.838822 (+ 36us) server_negotiation.cc:407] Connection header received
0925 10:55:05.839016 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:05.839020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:05.839110 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:05.839248 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:55:05.839633 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:05.840392 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:05.841809 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:05.842202 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:05.842317 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:05.842787 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:55:05.843357 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:55:05.843468 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:05.843572 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 10:55:05.843642 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":214,"threads_started":1}
I20260925 10:55:05.852128 6619 raft_consensus.cc:1261] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 2 FOLLOWER]: Refusing update from remote peer 5017255eb8f34c7abd7e4ec3f63dc84f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260925 10:55:05.852126 6905 raft_consensus.cc:1261] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [term 2 FOLLOWER]: Refusing update from remote peer 5017255eb8f34c7abd7e4ec3f63dc84f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260925 10:55:05.852936 7005 consensus_queue.cc:1035] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:05.853283 7004 consensus_queue.cc:1035] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:05.865932 7020 mvcc.cc:205] Tried to move back new op lower bound from 7333206859160600576 to 7333206857410904064. Current Snapshot: MvccSnapshot[applied={T|T < 7333206859160600576}]
I20260925 10:55:05.900921 5697 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:55:05.901219 5697 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:55:05.907601 5697 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44235815436447c3ab94d41adcbe92c5: Sending DeleteTablet for 3 replicas of tablet eed11e666ba642cc89187713a19f175f
I20260925 10:55:05.909781 6744 tablet_service.cc:1514] Processing DeleteTablet for tablet eed11e666ba642cc89187713a19f175f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:05 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:47650
I20260925 10:55:05.909781 6885 tablet_service.cc:1514] Processing DeleteTablet for tablet eed11e666ba642cc89187713a19f175f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:05 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:37846
I20260925 10:55:05.910025 6599 tablet_service.cc:1514] Processing DeleteTablet for tablet eed11e666ba642cc89187713a19f175f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:05 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:44474
I20260925 10:55:05.920707 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 5665
I20260925 10:55:05.921497 7033 tablet_replica.cc:330] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc: stopping tablet replica
I20260925 10:55:05.921934 7032 tablet_replica.cc:330] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f: stopping tablet replica
I20260925 10:55:05.922799 7032 raft_consensus.cc:2229] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 2 LEADER]: Raft consensus shutting down.
I20260925 10:55:05.923167 7033 raft_consensus.cc:2229] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:05.924142 5727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:05.926219 7034 tablet_replica.cc:330] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11: stopping tablet replica
I20260925 10:55:05.926694 7034 raft_consensus.cc:2229] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:05.927214 7034 raft_consensus.cc:2258] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:05.929584 7034 ts_tablet_manager.cc:1905] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:05.930433 7033 raft_consensus.cc:2258] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:05.931030 7032 raft_consensus.cc:2258] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:05.933990 7033 ts_tablet_manager.cc:1905] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:05.935998 7032 ts_tablet_manager.cc:1905] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:05.936568 7034 ts_tablet_manager.cc:1918] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260925 10:55:05.936755 7034 log.cc:1196] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/eed11e666ba642cc89187713a19f175f
I20260925 10:55:05.937299 7034 ts_tablet_manager.cc:1939] T eed11e666ba642cc89187713a19f175f P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Deleting consensus metadata
I20260925 10:55:05.948004 7033 ts_tablet_manager.cc:1918] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260925 10:55:05.948185 7033 log.cc:1196] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/eed11e666ba642cc89187713a19f175f
I20260925 10:55:05.949369 7033 ts_tablet_manager.cc:1939] T eed11e666ba642cc89187713a19f175f P f727914630b045db8981c6190cd467dc: Deleting consensus metadata
I20260925 10:55:05.950450 7032 ts_tablet_manager.cc:1918] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260925 10:55:05.950701 7032 log.cc:1196] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/eed11e666ba642cc89187713a19f175f
I20260925 10:55:05.951452 7032 ts_tablet_manager.cc:1939] T eed11e666ba642cc89187713a19f175f P 5017255eb8f34c7abd7e4ec3f63dc84f: Deleting consensus metadata
I20260925 10:55:05.964811 7036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:05.914903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39402 (local address 127.31.62.189:36481)
0925 10:55:05.942637 (+ 27734us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:05.942643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:05.942679 (+ 36us) server_negotiation.cc:407] Connection header received
0925 10:55:05.942746 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:05.942751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:05.942841 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:05.945062 (+ 2221us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:05.945279 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:05.946385 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:05.948345 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:05.951579 (+ 3234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:05.952435 (+ 856us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:05.959214 (+ 6779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:05.959220 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:05.959223 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:05.959264 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:05.962032 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:05.962838 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:05.962843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:05.962845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:05.962989 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:05.963390 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:05.963394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:05.963396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:05.963665 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:05.963884 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:05.964239 (+ 355us) server_negotiation.cc:299] Negotiation successful
0925 10:55:05.964451 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27618,"thread_start_us":34548,"threads_started":1}
I20260925 10:55:05.977252 7037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:05.920048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51014 (local address 127.31.62.188:46577)
0925 10:55:05.942180 (+ 22132us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:05.942187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:05.965017 (+ 22830us) server_negotiation.cc:407] Connection header received
0925 10:55:05.965250 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:05.965256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:05.965338 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:05.965416 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:05.967117 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:05.967898 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:05.969229 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:05.969646 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:05.969760 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:05.972195 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:05.972200 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:05.972203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:05.972237 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:05.974777 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:05.975392 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:05.975396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:05.975399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:05.975520 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:05.975865 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:05.975870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:05.975873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:05.976132 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:05.976356 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:05.976646 (+ 290us) server_negotiation.cc:299] Negotiation successful
0925 10:55:05.976891 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21966,"thread_start_us":22154,"threads_started":1}
I20260925 10:55:06.114253 5681 catalog_manager.cc:4841] TS 5017255eb8f34c7abd7e4ec3f63dc84f (127.31.62.130:44351): tablet eed11e666ba642cc89187713a19f175f (table test-table [id=a66618cebabb4557bc779204c5b1d1e3]) successfully deleted
I20260925 10:55:06.115015 5683 catalog_manager.cc:4841] TS f727914630b045db8981c6190cd467dc (127.31.62.131:34571): tablet eed11e666ba642cc89187713a19f175f (table test-table [id=a66618cebabb4557bc779204c5b1d1e3]) successfully deleted
I20260925 10:55:06.115741 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 5665
I20260925 10:55:06.115732 5682 catalog_manager.cc:4841] TS 1a0685c684ef4ad2a1d95ef3cf1f2a11 (127.31.62.129:34941): tablet eed11e666ba642cc89187713a19f175f (table test-table [id=a66618cebabb4557bc779204c5b1d1e3]) successfully deleted
I20260925 10:55:06.169376 7037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:06.137519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51028 (local address 127.31.62.188:46577)
0925 10:55:06.137733 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:06.137739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:06.151597 (+ 13858us) server_negotiation.cc:407] Connection header received
0925 10:55:06.151942 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:06.151951 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:06.152062 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:06.152155 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:06.155795 (+ 3640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:06.156806 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:06.158899 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:06.159346 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:06.159469 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:06.162819 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:06.162824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:06.162828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:06.162863 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:06.166218 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:06.167080 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:06.167084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:06.167086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:06.167258 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:06.167705 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:06.167711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:06.167715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:06.168099 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:06.168362 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:06.168699 (+ 337us) server_negotiation.cc:299] Negotiation successful
0925 10:55:06.168971 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 10:55:06.171622 7036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:06.137530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39414 (local address 127.31.62.189:36481)
0925 10:55:06.137785 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:06.137792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:06.155636 (+ 17844us) server_negotiation.cc:407] Connection header received
0925 10:55:06.156258 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:06.156263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:06.156345 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:06.156459 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:06.158882 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:06.159942 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:06.161422 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:06.161890 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:06.162024 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:06.165371 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:06.165376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:06.165380 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:06.165417 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:06.168562 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:06.169449 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:06.169455 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:06.169459 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:06.169618 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:06.170076 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:06.170082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:06.170086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:06.170441 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:06.170730 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:06.171067 (+ 337us) server_negotiation.cc:299] Negotiation successful
0925 10:55:06.171291 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260925 10:55:06.886382 7046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:06.885648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.130:33605)
0925 10:55:06.886093 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:06.886228 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":231,"threads_started":1}
W20260925 10:55:06.886755 6810 heartbeater.cc:644] Failed to heartbeat to 127.31.62.190:40377 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
I20260925 10:55:06.893100 7047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:06.892156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.131:39799)
0925 10:55:06.892618 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:06.892755 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":224,"threads_started":1}
W20260925 10:55:06.893591 6951 heartbeater.cc:644] Failed to heartbeat to 127.31.62.190:40377 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
I20260925 10:55:06.894877 7048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:06.893802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.129:55351)
0925 10:55:06.894582 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:06.894713 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":568,"threads_started":1}
W20260925 10:55:06.895213 6665 heartbeater.cc:644] Failed to heartbeat to 127.31.62.190:40377 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
I20260925 10:55:07.408555 7049 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 44235815436447c3ab94d41adcbe92c5)
I20260925 10:55:07.408830 7050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 44235815436447c3ab94d41adcbe92c5)
I20260925 10:55:07.408924 7049 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } }
I20260925 10:55:07.409116 7050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } }
I20260925 10:55:07.410409 7049 leader_election.cc:288] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44235815436447c3ab94d41adcbe92c5 (127.31.62.190:40377), 089603cb4c1d4455b174b99ade00cdf7 (127.31.62.189:36481)
I20260925 10:55:07.410429 7050 leader_election.cc:288] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44235815436447c3ab94d41adcbe92c5 (127.31.62.190:40377), e6a2bc1541614ee3b1c9f430a8e74994 (127.31.62.188:46577)
I20260925 10:55:07.419035 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.410582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:52464)
0925 10:55:07.417839 (+ 7257us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.418064 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7117,"threads_started":1}
W20260925 10:55:07.420285 6284 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
I20260925 10:55:07.421299 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.410583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:52462)
0925 10:55:07.420798 (+ 10215us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.421054 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10103,"threads_started":1}
W20260925 10:55:07.426146 6471 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
I20260925 10:55:07.430261 7052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.429676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:52488)
0925 10:55:07.429873 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.430043 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260925 10:55:07.430701 6284 leader_election.cc:334] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 44235815436447c3ab94d41adcbe92c5 (127.31.62.190:40377): Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
I20260925 10:55:07.430796 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.410879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:36481 (local address 127.0.0.1:39420)
0925 10:55:07.421525 (+ 10646us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.421544 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:07.421695 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:07.422292 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:07.422297 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:07.422325 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:07.422668 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:07.422682 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.424599 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.424606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:07.427555 (+ 2949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:07.427579 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.430024 (+ 2445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.430030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:07.430217 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:07.430265 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 10:55:07.430354 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 10:55:07.430499 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10556,"thread_start_us":9665,"threads_started":1}
I20260925 10:55:07.431509 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.411236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39420 (local address 127.31.62.189:36481)
0925 10:55:07.420403 (+ 9167us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:07.420409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:07.421807 (+ 1398us) server_negotiation.cc:407] Connection header received
0925 10:55:07.421885 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:07.421891 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:07.421963 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:07.422053 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:07.422859 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.424402 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.427810 (+ 3408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.429672 (+ 1862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.429764 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:07.429864 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:07.430983 (+ 1119us) server_negotiation.cc:299] Negotiation successful
0925 10:55:07.431124 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9050,"thread_start_us":8900,"threads_started":1}
I20260925 10:55:07.432158 6307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "089603cb4c1d4455b174b99ade00cdf7" is_pre_election: true
I20260925 10:55:07.432566 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.432163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:52504)
0925 10:55:07.432332 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.432421 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260925 10:55:07.432567 6307 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6a2bc1541614ee3b1c9f430a8e74994 in term 1.
W20260925 10:55:07.432791 6471 leader_election.cc:334] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 44235815436447c3ab94d41adcbe92c5 (127.31.62.190:40377): Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
I20260925 10:55:07.433091 6469 leader_election.cc:302] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 089603cb4c1d4455b174b99ade00cdf7, e6a2bc1541614ee3b1c9f430a8e74994; no voters: 44235815436447c3ab94d41adcbe92c5
I20260925 10:55:07.433457 7049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 10:55:07.433596 7049 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 44235815436447c3ab94d41adcbe92c5)
I20260925 10:55:07.433697 7049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 1 FOLLOWER]: Advancing to term 2
I20260925 10:55:07.435488 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.411225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:46577 (local address 127.0.0.1:51034)
0925 10:55:07.419417 (+ 8192us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.419440 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:07.419614 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:07.428296 (+ 8682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:07.428301 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:07.428349 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:07.428698 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:07.428710 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.431334 (+ 2624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.431339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:07.433410 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:07.433442 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.434779 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.434785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:07.435009 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:07.435036 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 10:55:07.435164 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 10:55:07.435292 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8081,"thread_start_us":7405,"threads_started":1}
I20260925 10:55:07.435655 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.420871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51034 (local address 127.31.62.188:46577)
0925 10:55:07.427940 (+ 7069us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:07.427946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:07.427970 (+ 24us) server_negotiation.cc:407] Connection header received
0925 10:55:07.428040 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:07.428045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:07.428138 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:07.428268 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:07.429570 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.431198 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.433580 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.434613 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.434728 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:07.434847 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:07.435191 (+ 344us) server_negotiation.cc:299] Negotiation successful
0925 10:55:07.435331 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6915,"thread_start_us":7583,"threads_started":1}
I20260925 10:55:07.436172 6494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "089603cb4c1d4455b174b99ade00cdf7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" is_pre_election: true
I20260925 10:55:07.436369 7049 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } }
I20260925 10:55:07.436760 6494 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 089603cb4c1d4455b174b99ade00cdf7 in current term 2: Already voted for candidate e6a2bc1541614ee3b1c9f430a8e74994 in this term.
I20260925 10:55:07.436997 7049 leader_election.cc:288] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [CANDIDATE]: Term 2 election: Requested vote from peers 44235815436447c3ab94d41adcbe92c5 (127.31.62.190:40377), 089603cb4c1d4455b174b99ade00cdf7 (127.31.62.189:36481)
I20260925 10:55:07.437541 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.437096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:52508)
0925 10:55:07.437234 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.437408 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 10:55:07.437554 6307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "089603cb4c1d4455b174b99ade00cdf7"
I20260925 10:55:07.437624 6281 leader_election.cc:302] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 089603cb4c1d4455b174b99ade00cdf7; no voters: 44235815436447c3ab94d41adcbe92c5, e6a2bc1541614ee3b1c9f430a8e74994
I20260925 10:55:07.437780 6307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 1 FOLLOWER]: Advancing to term 2
I20260925 10:55:07.438728 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.438377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:52514)
0925 10:55:07.438557 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.438623 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260925 10:55:07.438886 6471 leader_election.cc:334] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 44235815436447c3ab94d41adcbe92c5 (127.31.62.190:40377): Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
I20260925 10:55:07.440021 6307 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6a2bc1541614ee3b1c9f430a8e74994 in term 2.
I20260925 10:55:07.440196 7050 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 10:55:07.440409 6469 leader_election.cc:302] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 089603cb4c1d4455b174b99ade00cdf7, e6a2bc1541614ee3b1c9f430a8e74994; no voters: 44235815436447c3ab94d41adcbe92c5
I20260925 10:55:07.440678 7049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 10:55:07.441109 7049 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [term 2 LEADER]: Becoming Leader. State: Replica: e6a2bc1541614ee3b1c9f430a8e74994, State: Running, Role: LEADER
I20260925 10:55:07.441478 7049 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } }
I20260925 10:55:07.443627 7059 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6a2bc1541614ee3b1c9f430a8e74994. Latest consensus state: current_term: 2 leader_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } } }
I20260925 10:55:07.443908 7059 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:07.444532 7061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:55:07.448786 7061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:55:07.449740 7061 catalog_manager.cc:1259] Loaded cluster ID: 86416e3acf79441b9253cc5f1538d331
I20260925 10:55:07.449831 7061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:55:07.450990 7061 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:55:07.451865 7061 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: Loaded TSK: 0
I20260925 10:55:07.452775 7061 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:55:07.504833 6484 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:55:07.512953 6484 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:55:07.517719 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.517319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:52546)
0925 10:55:07.517491 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.517573 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 10:55:07.517763 6307 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [term 2 FOLLOWER]: Refusing update from remote peer e6a2bc1541614ee3b1c9f430a8e74994: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260925 10:55:07.518558 7059 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [LEADER]: Connected to new peer: Peer: permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260925 10:55:07.518800 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.518396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:52558)
0925 10:55:07.518533 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.518638 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 10:55:07.518985 6471 consensus_peers.cc:487] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 -> Peer 44235815436447c3ab94d41adcbe92c5 (127.31.62.190:40377): Couldn't send request to peer 44235815436447c3ab94d41adcbe92c5. Status: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 10:55:07.521376 7050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6a2bc1541614ee3b1c9f430a8e74994. Latest consensus state: current_term: 2 leader_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } } }
I20260925 10:55:07.521646 7050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 10:55:07.523097 7059 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } } }
I20260925 10:55:07.523397 7059 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:07.524173 7050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "44235815436447c3ab94d41adcbe92c5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 40377 } } peers { permanent_uuid: "089603cb4c1d4455b174b99ade00cdf7" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 36481 } attrs { promote: false } } peers { permanent_uuid: "e6a2bc1541614ee3b1c9f430a8e74994" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 46577 } attrs { promote: false } } }
I20260925 10:55:07.526736 7050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089603cb4c1d4455b174b99ade00cdf7 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 10:55:07.541389 7051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.532280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34941 (local address 127.0.0.1:49444)
0925 10:55:07.532683 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.532702 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:07.532815 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:07.533587 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:07.533592 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:07.533615 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:07.534070 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:07.534096 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.536157 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.536163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:07.538510 (+ 2347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:07.538528 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.540397 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.540404 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:07.540628 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:07.540671 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 10:55:07.540745 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 10:55:07.540988 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246}
I20260925 10:55:07.542819 7068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.532394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49444 (local address 127.31.62.129:34941)
0925 10:55:07.533255 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:07.533260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:07.533279 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:55:07.533344 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:07.533350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:07.533436 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:07.534629 (+ 1193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:07.534819 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.535981 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.538776 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.540121 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.542167 (+ 2046us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:07.542289 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:07.542346 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 10:55:07.542501 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":875,"threads_started":1}
I20260925 10:55:07.543583 6599 tablet_service.cc:1467] Processing CreateTablet for tablet 047378a4db6d46b0af9001326bafa4de (DEFAULT_TABLE table=test-table [id=81a55e9adf334289bc4e8466d8fd4d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:07.544127 6599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 047378a4db6d46b0af9001326bafa4de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:07.551208 7072 tablet_bootstrap.cc:492] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Bootstrap starting.
I20260925 10:55:07.552881 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.533061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53324 (local address 127.31.62.130:44351)
0925 10:55:07.546174 (+ 13113us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:07.546180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:07.546204 (+ 24us) server_negotiation.cc:407] Connection header received
0925 10:55:07.546301 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:07.546307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:07.546388 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:07.547039 (+ 651us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:07.547429 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.548473 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.550778 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.551858 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.551977 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:07.552070 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:07.552458 (+ 388us) server_negotiation.cc:299] Negotiation successful
0925 10:55:07.552600 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12954,"thread_start_us":13799,"threads_started":1}
I20260925 10:55:07.552907 7051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.533056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:44351 (local address 127.0.0.1:53324)
0925 10:55:07.541684 (+ 8628us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.541707 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:07.541832 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:07.547078 (+ 5246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:07.547084 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:07.547112 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:07.547278 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:07.547287 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.548651 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.548656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:07.550604 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:07.550618 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.552004 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.552010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:07.552219 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:07.552297 (+ 78us) client_negotiation.cc:769] Sending connection context
0925 10:55:07.552376 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 10:55:07.552518 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8503,"mutex_wait_us":75,"thread_start_us":13588,"threads_started":1}
I20260925 10:55:07.553093 7072 tablet_bootstrap.cc:654] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:07.553646 6744 tablet_service.cc:1467] Processing CreateTablet for tablet 047378a4db6d46b0af9001326bafa4de (DEFAULT_TABLE table=test-table [id=81a55e9adf334289bc4e8466d8fd4d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:07.554083 6744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 047378a4db6d46b0af9001326bafa4de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:07.554493 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.532481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:34571 (local address 127.0.0.1:58606)
0925 10:55:07.532713 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.532731 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:07.532835 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:07.549216 (+ 16381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:07.549221 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:07.549248 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:07.549417 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:07.549427 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.550378 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.550383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:07.552294 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:07.552309 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.553798 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.553804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:07.553971 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:07.554007 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 10:55:07.554101 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 10:55:07.554231 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":112}
I20260925 10:55:07.555117 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.533214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58606 (local address 127.31.62.131:34571)
0925 10:55:07.548891 (+ 15677us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:07.548897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:07.548927 (+ 30us) server_negotiation.cc:407] Connection header received
0925 10:55:07.549006 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:07.549012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:07.549083 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:07.549178 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:07.549553 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.550240 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.552479 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.553632 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.553759 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:07.553857 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:07.554692 (+ 835us) server_negotiation.cc:299] Negotiation successful
0925 10:55:07.554834 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15558,"thread_start_us":15416,"threads_started":1}
I20260925 10:55:07.555780 6885 tablet_service.cc:1467] Processing CreateTablet for tablet 047378a4db6d46b0af9001326bafa4de (DEFAULT_TABLE table=test-table [id=81a55e9adf334289bc4e8466d8fd4d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:07.556159 6885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 047378a4db6d46b0af9001326bafa4de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:07.556782 7072 tablet_bootstrap.cc:492] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11: No bootstrap required, opened a new log
I20260925 10:55:07.556960 7072 ts_tablet_manager.cc:1397] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260925 10:55:07.557742 7072 raft_consensus.cc:348] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } } peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } }
I20260925 10:55:07.557982 7072 raft_consensus.cc:374] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:07.558069 7072 raft_consensus.cc:729] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a0685c684ef4ad2a1d95ef3cf1f2a11, State: Initialized, Role: FOLLOWER
I20260925 10:55:07.558324 7072 consensus_queue.cc:260] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } } peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } }
I20260925 10:55:07.565367 7072 ts_tablet_manager.cc:1428] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260925 10:55:07.565673 6664 heartbeater.cc:503] Master 127.31.62.188:46577 was elected leader, sending a full tablet report...
I20260925 10:55:07.568325 7077 tablet_bootstrap.cc:492] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc: Bootstrap starting.
I20260925 10:55:07.570124 7077 tablet_bootstrap.cc:654] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:07.571831 7075 tablet_bootstrap.cc:492] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f: Bootstrap starting.
I20260925 10:55:07.572415 7076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.559460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.129:43591)
0925 10:55:07.571977 (+ 12517us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.572126 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12386,"thread_start_us":12236,"threads_started":1}
I20260925 10:55:07.573570 7076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.573058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.129:58051)
0925 10:55:07.573339 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.573429 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 10:55:07.573645 7075 tablet_bootstrap.cc:654] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:07.574173 7077 tablet_bootstrap.cc:492] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc: No bootstrap required, opened a new log
I20260925 10:55:07.574314 7077 ts_tablet_manager.cc:1397] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260925 10:55:07.574944 7077 raft_consensus.cc:348] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } } peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } }
I20260925 10:55:07.575168 7077 raft_consensus.cc:374] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:07.575256 7077 raft_consensus.cc:729] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f727914630b045db8981c6190cd467dc, State: Initialized, Role: FOLLOWER
I20260925 10:55:07.575524 7075 tablet_bootstrap.cc:492] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f: No bootstrap required, opened a new log
I20260925 10:55:07.575488 7077 consensus_queue.cc:260] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } } peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } }
I20260925 10:55:07.575657 7075 ts_tablet_manager.cc:1397] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 10:55:07.576195 7077 ts_tablet_manager.cc:1428] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 10:55:07.576416 7075 raft_consensus.cc:348] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } } peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } }
I20260925 10:55:07.576632 7075 raft_consensus.cc:374] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:07.576721 7075 raft_consensus.cc:729] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5017255eb8f34c7abd7e4ec3f63dc84f, State: Initialized, Role: FOLLOWER
I20260925 10:55:07.578789 6950 heartbeater.cc:503] Master 127.31.62.188:46577 was elected leader, sending a full tablet report...
I20260925 10:55:07.577039 7075 consensus_queue.cc:260] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } } peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } }
I20260925 10:55:07.580314 7075 ts_tablet_manager.cc:1428] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260925 10:55:07.581790 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.580918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.130:41997)
0925 10:55:07.581388 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.581527 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"threads_started":1}
I20260925 10:55:07.583361 6809 heartbeater.cc:503] Master 127.31.62.188:46577 was elected leader, sending a full tablet report...
I20260925 10:55:07.584020 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.583616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.130:41677)
0925 10:55:07.583772 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.583873 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260925 10:55:07.584381 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.576558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.131:56305)
0925 10:55:07.583968 (+ 7410us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.584121 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7298,"thread_start_us":6569,"threads_started":1}
I20260925 10:55:07.585479 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.584950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.131:33767)
0925 10:55:07.585254 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.585342 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"threads_started":1}
I20260925 10:55:07.846789 7082 raft_consensus.cc:482] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:55:07.847008 7082 raft_consensus.cc:504] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } } peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } }
I20260925 10:55:07.847689 7082 leader_election.cc:288] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a0685c684ef4ad2a1d95ef3cf1f2a11 (127.31.62.129:34941), f727914630b045db8981c6190cd467dc (127.31.62.131:34571)
I20260925 10:55:07.848320 6619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "047378a4db6d46b0af9001326bafa4de" candidate_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" is_pre_election: true
I20260925 10:55:07.848308 6905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "047378a4db6d46b0af9001326bafa4de" candidate_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f727914630b045db8981c6190cd467dc" is_pre_election: true
I20260925 10:55:07.848560 6619 raft_consensus.cc:2454] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5017255eb8f34c7abd7e4ec3f63dc84f in term 0.
I20260925 10:55:07.848593 6905 raft_consensus.cc:2454] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5017255eb8f34c7abd7e4ec3f63dc84f in term 0.
I20260925 10:55:07.849120 6698 leader_election.cc:302] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a0685c684ef4ad2a1d95ef3cf1f2a11, 5017255eb8f34c7abd7e4ec3f63dc84f; no voters:
I20260925 10:55:07.849565 7082 raft_consensus.cc:2781] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:55:07.849726 7082 raft_consensus.cc:482] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:55:07.849817 7082 raft_consensus.cc:3037] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:07.852061 7082 raft_consensus.cc:504] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } } peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } }
I20260925 10:55:07.852676 7082 leader_election.cc:288] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [CANDIDATE]: Term 1 election: Requested vote from peers 1a0685c684ef4ad2a1d95ef3cf1f2a11 (127.31.62.129:34941), f727914630b045db8981c6190cd467dc (127.31.62.131:34571)
I20260925 10:55:07.853327 6905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "047378a4db6d46b0af9001326bafa4de" candidate_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f727914630b045db8981c6190cd467dc"
I20260925 10:55:07.853343 6619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "047378a4db6d46b0af9001326bafa4de" candidate_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11"
I20260925 10:55:07.853559 6619 raft_consensus.cc:3037] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:07.853576 6905 raft_consensus.cc:3037] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:07.856542 6619 raft_consensus.cc:2454] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5017255eb8f34c7abd7e4ec3f63dc84f in term 1.
I20260925 10:55:07.856527 6905 raft_consensus.cc:2454] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5017255eb8f34c7abd7e4ec3f63dc84f in term 1.
I20260925 10:55:07.857081 6699 leader_election.cc:302] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1a0685c684ef4ad2a1d95ef3cf1f2a11, 5017255eb8f34c7abd7e4ec3f63dc84f, f727914630b045db8981c6190cd467dc; no voters:
I20260925 10:55:07.857393 7082 raft_consensus.cc:2781] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:55:07.857767 7082 raft_consensus.cc:686] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [term 1 LEADER]: Becoming Leader. State: Replica: 5017255eb8f34c7abd7e4ec3f63dc84f, State: Running, Role: LEADER
I20260925 10:55:07.858037 7082 consensus_queue.cc:237] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } } peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } }
I20260925 10:55:07.858911 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.858448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.130:45453)
0925 10:55:07.858619 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.858728 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260925 10:55:07.861147 6484 catalog_manager.cc:5495] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5017255eb8f34c7abd7e4ec3f63dc84f (127.31.62.130). New cstate: current_term: 1 leader_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5017255eb8f34c7abd7e4ec3f63dc84f" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44351 } health_report { overall_health: HEALTHY } } }
I20260925 10:55:07.908687 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.904236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53336 (local address 127.31.62.130:44351)
0925 10:55:07.904408 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:07.904412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:07.904430 (+ 18us) server_negotiation.cc:407] Connection header received
0925 10:55:07.904502 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:07.904505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:07.904567 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:07.904632 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:55:07.905019 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.905751 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.907011 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.907457 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.907549 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:07.907960 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:55:07.908061 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:55:07.908120 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:07.908301 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 10:55:07.908373 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 10:55:07.911990 6905 raft_consensus.cc:1261] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc [term 1 FOLLOWER]: Refusing update from remote peer 5017255eb8f34c7abd7e4ec3f63dc84f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:55:07.911995 6619 raft_consensus.cc:1261] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 1 FOLLOWER]: Refusing update from remote peer 5017255eb8f34c7abd7e4ec3f63dc84f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:55:07.912621 7085 consensus_queue.cc:1035] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f727914630b045db8981c6190cd467dc" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:07.912925 7082 consensus_queue.cc:1035] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a0685c684ef4ad2a1d95ef3cf1f2a11" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:07.915545 7076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.914937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.129:51161)
0925 10:55:07.915192 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.915324 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260925 10:55:07.916491 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.915985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.131:59413)
0925 10:55:07.916177 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.916301 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260925 10:55:07.923384 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.922698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.130:32877)
0925 10:55:07.923097 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.923215 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295}
I20260925 10:55:07.923964 7076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.923404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.129:39901)
0925 10:55:07.923701 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.923792 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195}
I20260925 10:55:07.926486 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.926010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.130:40023)
0925 10:55:07.926161 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.926276 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 10:55:07.929199 6483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:55:07.929548 6483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:55:07.929912 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.929340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.31.62.131:50493)
0925 10:55:07.929634 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:07.929728 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184}
I20260925 10:55:07.933241 7090 mvcc.cc:205] Tried to move back new op lower bound from 7333206867601371136 to 7333206867389222912. Current Snapshot: MvccSnapshot[applied={T|T < 7333206867601371136}]
I20260925 10:55:07.937112 6483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6a2bc1541614ee3b1c9f430a8e74994: Sending DeleteTablet for 3 replicas of tablet 047378a4db6d46b0af9001326bafa4de
I20260925 10:55:07.938129 6599 tablet_service.cc:1514] Processing DeleteTablet for tablet 047378a4db6d46b0af9001326bafa4de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:07 UTC) from {username='oryx', principal='oryx/127.31.62.188@KRBTEST.COM'} at 127.0.0.1:49444
I20260925 10:55:07.938452 6885 tablet_service.cc:1514] Processing DeleteTablet for tablet 047378a4db6d46b0af9001326bafa4de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:07 UTC) from {username='oryx', principal='oryx/127.31.62.188@KRBTEST.COM'} at 127.0.0.1:58606
I20260925 10:55:07.938550 6744 tablet_service.cc:1514] Processing DeleteTablet for tablet 047378a4db6d46b0af9001326bafa4de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:07 UTC) from {username='oryx', principal='oryx/127.31.62.188@KRBTEST.COM'} at 127.0.0.1:53324
I20260925 10:55:07.938680 6884 tablet_service.cc:1514] Processing DeleteTablet for tablet 047378a4db6d46b0af9001326bafa4de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:07 UTC) from {username='oryx', principal='oryx/127.31.62.188@KRBTEST.COM'} at 127.0.0.1:58606
I20260925 10:55:07.939222 7097 tablet_replica.cc:330] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc: stopping tablet replica
I20260925 10:55:07.939407 7097 raft_consensus.cc:2229] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:07.939692 7097 raft_consensus.cc:2258] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:07.943540 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 6456
I20260925 10:55:07.946084 7097 ts_tablet_manager.cc:1905] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:07.951750 7096 tablet_replica.cc:330] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11: stopping tablet replica
I20260925 10:55:07.951992 7096 raft_consensus.cc:2229] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:07.952371 7096 raft_consensus.cc:2258] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:07.954663 7096 ts_tablet_manager.cc:1905] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:07.955994 7097 ts_tablet_manager.cc:1918] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:55:07.956151 7097 log.cc:1196] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/047378a4db6d46b0af9001326bafa4de
I20260925 10:55:07.956615 7097 ts_tablet_manager.cc:1939] T 047378a4db6d46b0af9001326bafa4de P f727914630b045db8981c6190cd467dc: Deleting consensus metadata
I20260925 10:55:07.957254 7098 tablet_replica.cc:330] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f: stopping tablet replica
I20260925 10:55:07.957413 7098 raft_consensus.cc:2229] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:55:07.957741 7098 raft_consensus.cc:2258] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:07.958729 7098 ts_tablet_manager.cc:1905] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:07.961215 7096 ts_tablet_manager.cc:1918] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:55:07.961362 7096 log.cc:1196] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/047378a4db6d46b0af9001326bafa4de
I20260925 10:55:07.961856 7096 ts_tablet_manager.cc:1939] T 047378a4db6d46b0af9001326bafa4de P 1a0685c684ef4ad2a1d95ef3cf1f2a11: Deleting consensus metadata
I20260925 10:55:07.963042 6469 catalog_manager.cc:4841] TS 1a0685c684ef4ad2a1d95ef3cf1f2a11 (127.31.62.129:34941): tablet 047378a4db6d46b0af9001326bafa4de (table test-table [id=81a55e9adf334289bc4e8466d8fd4d63]) successfully deleted
I20260925 10:55:07.963809 7098 ts_tablet_manager.cc:1918] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:55:07.963865 6470 catalog_manager.cc:4841] TS f727914630b045db8981c6190cd467dc (127.31.62.131:34571): tablet 047378a4db6d46b0af9001326bafa4de (table test-table [id=81a55e9adf334289bc4e8466d8fd4d63]) successfully deleted
I20260925 10:55:07.963944 7098 log.cc:1196] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/047378a4db6d46b0af9001326bafa4de
I20260925 10:55:07.964372 7098 ts_tablet_manager.cc:1939] T 047378a4db6d46b0af9001326bafa4de P 5017255eb8f34c7abd7e4ec3f63dc84f: Deleting consensus metadata
W20260925 10:55:07.964509 6470 catalog_manager.cc:4808] TS f727914630b045db8981c6190cd467dc (127.31.62.131:34571): delete failed for tablet 047378a4db6d46b0af9001326bafa4de because the tablet was not found. No further retry: Not found: Tablet not found: 047378a4db6d46b0af9001326bafa4de
I20260925 10:55:07.965374 6468 catalog_manager.cc:4841] TS 5017255eb8f34c7abd7e4ec3f63dc84f (127.31.62.130:44351): tablet 047378a4db6d46b0af9001326bafa4de (table test-table [id=81a55e9adf334289bc4e8466d8fd4d63]) successfully deleted
I20260925 10:55:07.968120 7068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:07.944881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49454 (local address 127.31.62.129:34941)
0925 10:55:07.945177 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:07.945184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:07.951531 (+ 6347us) server_negotiation.cc:407] Connection header received
0925 10:55:07.951725 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:07.951733 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:07.951801 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:07.951891 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:07.953653 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.954686 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.956805 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:07.957496 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:07.957599 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:07.961374 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:07.961396 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:07.961420 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:07.961475 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:07.965279 (+ 3804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:07.966043 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:07.966051 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:07.966053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:07.966195 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:07.966598 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:07.966602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:07.966604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:07.966969 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:07.967263 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:07.967554 (+ 291us) server_negotiation.cc:299] Negotiation successful
0925 10:55:07.967794 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260925 10:55:07.968760 6659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:08.063338 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 6456
I20260925 10:55:08.078279 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 6676
I20260925 10:55:08.093855 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:08.079101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53352 (local address 127.31.62.130:44351)
0925 10:55:08.079281 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:08.079287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:08.079308 (+ 21us) server_negotiation.cc:407] Connection header received
0925 10:55:08.079508 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:08.079513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:08.079615 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:08.079703 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:08.081130 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:08.081853 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:08.083298 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:08.083717 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:08.083838 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:08.087315 (+ 3477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:08.087335 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:08.087352 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:08.087413 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:08.090722 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:08.091586 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:08.091595 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:08.091598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:08.091759 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:08.092191 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:08.092195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:08.092198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:08.092592 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:08.092924 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:08.093269 (+ 345us) server_negotiation.cc:299] Negotiation successful
0925 10:55:08.093489 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 10:55:08.094631 6804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:08.131942 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:08.131193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:52572)
0925 10:55:08.131671 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:08.131773 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":244,"threads_started":1}
I20260925 10:55:08.133332 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:08.132975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:40377 (local address 127.0.0.1:52580)
0925 10:55:08.133112 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:08.133200 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 10:55:08.212628 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 6676
I20260925 10:55:08.232791 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 6822
I20260925 10:55:08.247179 7104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:08.233583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58608 (local address 127.31.62.131:34571)
0925 10:55:08.234073 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:08.234078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:08.234097 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:55:08.234148 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:08.234152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:08.234212 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:08.234295 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:08.235655 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:08.236389 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:08.237713 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:08.238181 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:08.238293 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:08.240742 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:08.240758 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:08.240775 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:08.240815 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:08.243680 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:08.244518 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:08.244526 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:08.244529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:08.244680 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:08.245462 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:08.245466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:08.245468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:08.245883 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:08.246123 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:08.246560 (+ 437us) server_negotiation.cc:299] Negotiation successful
0925 10:55:08.246779 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":251,"threads_started":1}
I20260925 10:55:08.247928 6945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:08.343813 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 6822
I20260925 10:55:08.359052 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 6158
I20260925 10:55:08.360203 6327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:08.479187 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 6158
I20260925 10:55:08.496673 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 6257
I20260925 10:55:08.497849 6514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:08.616669 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 6257
2026-09-25T10:55:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12041 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:55:08 dist-test-slave-nx7s krb5kdc[7110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:55:08 dist-test-slave-nx7s krb5kdc[7110](info): set up 2 sockets
Sep 25 10:55:08 dist-test-slave-nx7s krb5kdc[7110](info): commencing operation
krb5kdc: starting...
W20260925 10:55:09.649735 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.001s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:55:09 dist-test-slave-nx7s krb5kdc[7110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:55:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:55:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.190.keytab.
Entry for principal oryx/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.190.keytab.
I20260925 10:55:09.869412 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:45847
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.190.keytab
--principal=oryx/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:46743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:45847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:55:10.193629 7123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:10.193954 7123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:10.194080 7123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:10.201699 7123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:55:10.201825 7123 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 10:55:10.201865 7123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:10.201946 7123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:10.202008 7123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:55:10.202102 7123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:55:10.212321 7123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:46743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:45847
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.190.keytab
--principal=oryx/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:45847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:10.213699 7123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:10.215387 7123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:10.228505 7128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:10.228447 7131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:10.228505 7129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:10.228935 7123 server_base.cc:1034] running on GCE node
I20260925 10:55:10.229794 7123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:10.231335 7123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:10.232577 7123 hybrid_clock.cc:648] HybridClock initialized: now 1790333710232516 us; error 37 us; skew 500 ppm
I20260925 10:55:10.233006 7123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:10 dist-test-slave-nx7s krb5kdc[7110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333710, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:10.236157 7123 init.cc:377] Logged in from keytab as oryx/127.31.62.190@KRBTEST.COM (short username oryx)
I20260925 10:55:10.237810 7123 webserver.cc:460] Webserver started at http://127.31.62.190:35121/ using document root <none> and password file <none>
I20260925 10:55:10.238559 7123 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:10.238664 7123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:10.239033 7123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:10.242332 7123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "b871a10fcdcd48c39c2ccb5b6c819927"
format_stamp: "Formatted at 2026-09-25 10:55:10 on dist-test-slave-nx7s"
server_key: "0390bac3001615c7868965050cdca71c"
server_key_iv: "6328adea3038ecef9c62d77d2760ed6e"
server_key_version: "encryptionkey@0"
I20260925 10:55:10.243230 7123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "b871a10fcdcd48c39c2ccb5b6c819927"
format_stamp: "Formatted at 2026-09-25 10:55:10 on dist-test-slave-nx7s"
server_key: "0390bac3001615c7868965050cdca71c"
server_key_iv: "6328adea3038ecef9c62d77d2760ed6e"
server_key_version: "encryptionkey@0"
I20260925 10:55:10.248997 7123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 10:55:10.252709 7138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:10.254390 7123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:55:10.254560 7123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "b871a10fcdcd48c39c2ccb5b6c819927"
format_stamp: "Formatted at 2026-09-25 10:55:10 on dist-test-slave-nx7s"
server_key: "0390bac3001615c7868965050cdca71c"
server_key_iv: "6328adea3038ecef9c62d77d2760ed6e"
server_key_version: "encryptionkey@0"
I20260925 10:55:10.254786 7123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:10.301456 7123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:10.305939 7123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:10.306233 7123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:10.326694 7123 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:45847
I20260925 10:55:10.326716 7199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:45847 every 8 connection(s)
I20260925 10:55:10.328056 7123 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:55:10.330914 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 7123
I20260925 10:55:10.331116 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:55:10.331761 31994 external_mini_cluster.cc:1428] Setting key 29ba90e92a3c3fedaca34f2f26f68d36
I20260925 10:55:10.332010 7200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:10.347334 7200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927: Bootstrap starting.
Sep 25 10:55:10 dist-test-slave-nx7s krb5kdc[7110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
I20260925 10:55:10.351261 7200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:10.352686 7200 log.cc:824] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:10.357049 7200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927: No bootstrap required, opened a new log
I20260925 10:55:10.359364 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:10.333825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54328 (local address 127.31.62.190:45847)
0925 10:55:10.334685 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:10.334700 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:10.334754 (+ 54us) server_negotiation.cc:407] Connection header received
0925 10:55:10.337129 (+ 2375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:10.337164 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:10.337816 (+ 652us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:10.338405 (+ 589us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:10.339772 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:10.341844 (+ 2072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:10.343192 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:10.344377 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:10.344784 (+ 407us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:10.348723 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:10.348735 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:10.348757 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:10.348824 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:10.352787 (+ 3963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:10.353837 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:10.353847 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:10.353857 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:10.354068 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:10.355004 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:10.355010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:10.355014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:10.355781 (+ 767us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:10.356071 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:10.356945 (+ 874us) server_negotiation.cc:299] Negotiation successful
0925 10:55:10.357691 (+ 746us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":368,"threads_started":1}
I20260925 10:55:10.366878 7200 raft_consensus.cc:348] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871a10fcdcd48c39c2ccb5b6c819927" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45847 } }
I20260925 10:55:10.367363 7200 raft_consensus.cc:374] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:10.367475 7200 raft_consensus.cc:729] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b871a10fcdcd48c39c2ccb5b6c819927, State: Initialized, Role: FOLLOWER
I20260925 10:55:10.368185 7200 consensus_queue.cc:260] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871a10fcdcd48c39c2ccb5b6c819927" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45847 } }
I20260925 10:55:10.368470 7200 raft_consensus.cc:388] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:55:10.368614 7200 raft_consensus.cc:482] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:55:10.368795 7200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:10.371867 7200 raft_consensus.cc:504] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871a10fcdcd48c39c2ccb5b6c819927" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45847 } }
I20260925 10:55:10.372529 7200 leader_election.cc:302] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b871a10fcdcd48c39c2ccb5b6c819927; no voters:
I20260925 10:55:10.373275 7200 leader_election.cc:288] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:55:10.373430 7205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:55:10.375602 7200 sys_catalog.cc:564] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:55:10.376036 7205 raft_consensus.cc:686] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [term 1 LEADER]: Becoming Leader. State: Replica: b871a10fcdcd48c39c2ccb5b6c819927, State: Running, Role: LEADER
I20260925 10:55:10.376706 7205 consensus_queue.cc:237] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871a10fcdcd48c39c2ccb5b6c819927" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45847 } }
I20260925 10:55:10.379715 7207 sys_catalog.cc:455] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b871a10fcdcd48c39c2ccb5b6c819927. Latest consensus state: current_term: 1 leader_uuid: "b871a10fcdcd48c39c2ccb5b6c819927" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871a10fcdcd48c39c2ccb5b6c819927" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45847 } } }
I20260925 10:55:10.380085 7207 sys_catalog.cc:458] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:10.380514 7206 sys_catalog.cc:455] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b871a10fcdcd48c39c2ccb5b6c819927" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b871a10fcdcd48c39c2ccb5b6c819927" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45847 } } }
I20260925 10:55:10.380795 7206 sys_catalog.cc:458] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:10.381074 7216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:55:10.391856 7216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:55:10.401911 7216 catalog_manager.cc:1347] Generated new cluster ID: 26c5979337d34340b9cce6a5721314c1
I20260925 10:55:10.402035 7216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:55:10.417008 7216 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:55:10.418987 7216 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:55:10.435269 7216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927: Generated new TSK 0
I20260925 10:55:10.436362 7216 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.129.keytab.
Entry for principal oryx/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.129.keytab.
I20260925 10:55:10.492916 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:45847
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.129.keytab
--principal=oryx/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:46743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:55:10.825637 7227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:10.825927 7227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:10.826022 7227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:10.833436 7227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:10.833562 7227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:10.833657 7227 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 10:55:10.833734 7227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:55:10.843727 7227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:46743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.129.keytab
--principal=oryx/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.62.190:45847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:10.845026 7227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:10.846333 7227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:10.858778 7232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:10.859527 7233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:10.859695 7235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:10.860301 7227 server_base.cc:1034] running on GCE node
I20260925 10:55:10.860986 7227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:10.861987 7227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:10.863212 7227 hybrid_clock.cc:648] HybridClock initialized: now 1790333710863142 us; error 41 us; skew 500 ppm
I20260925 10:55:10.863513 7227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:10 dist-test-slave-nx7s krb5kdc[7110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333710, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:10.866679 7227 init.cc:377] Logged in from keytab as oryx/127.31.62.129@KRBTEST.COM (short username oryx)
I20260925 10:55:10.868120 7227 webserver.cc:460] Webserver started at http://127.31.62.129:36605/ using document root <none> and password file <none>
I20260925 10:55:10.868804 7227 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:10.868927 7227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:10.869210 7227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:10.872097 7227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "5b669bbc2aa449b7baf228a3a5c50f6d"
format_stamp: "Formatted at 2026-09-25 10:55:10 on dist-test-slave-nx7s"
server_key: "b7b298a09ba982a8ad7ce1eb65a85c5b"
server_key_iv: "70dd001328cc469c2412f7d216d6f67b"
server_key_version: "encryptionkey@0"
I20260925 10:55:10.872982 7227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "5b669bbc2aa449b7baf228a3a5c50f6d"
format_stamp: "Formatted at 2026-09-25 10:55:10 on dist-test-slave-nx7s"
server_key: "b7b298a09ba982a8ad7ce1eb65a85c5b"
server_key_iv: "70dd001328cc469c2412f7d216d6f67b"
server_key_version: "encryptionkey@0"
I20260925 10:55:10.878631 7227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:55:10.882576 7242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:10.884246 7227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:55:10.884418 7227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "5b669bbc2aa449b7baf228a3a5c50f6d"
format_stamp: "Formatted at 2026-09-25 10:55:10 on dist-test-slave-nx7s"
server_key: "b7b298a09ba982a8ad7ce1eb65a85c5b"
server_key_iv: "70dd001328cc469c2412f7d216d6f67b"
server_key_version: "encryptionkey@0"
I20260925 10:55:10.884639 7227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:10.905577 7227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:10.909976 7227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:10.910240 7227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:10.913172 7227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:10.913255 7227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:10.913374 7227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:10.913441 7227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 10:55:10 dist-test-slave-nx7s krb5kdc[7110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333710, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
I20260925 10:55:10.937536 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:10.913679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:35501 (local address 127.31.62.190:45847)
0925 10:55:10.916414 (+ 2735us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:10.916418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:10.916437 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:55:10.918605 (+ 2168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:10.918611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:10.918706 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:10.918853 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:10.921185 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:10.922341 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:10.924668 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:10.925359 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:10.925502 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:10.930110 (+ 4608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:10.930118 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:10.930124 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:10.930176 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:10.933804 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:10.934976 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:10.934983 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:10.934986 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:10.935158 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:10.935683 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:10.935689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:10.935692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:10.935970 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:10.936104 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:10.936931 (+ 827us) server_negotiation.cc:299] Negotiation successful
0925 10:55:10.937161 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2638,"thread_start_us":2550,"threads_started":1}
I20260925 10:55:10.938885 7252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:10.914520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:45847 (local address 127.31.62.129:35501)
0925 10:55:10.915494 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:10.915565 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:10.918149 (+ 2584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:10.919123 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:10.919132 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:10.919964 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:10.920918 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:10.920968 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:10.922563 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:10.922572 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:10.924447 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:10.924467 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:10.925641 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:10.925650 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:10.925943 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:10.926815 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:10.926858 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:10.929783 (+ 2925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:10.933994 (+ 4211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:10.934008 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:10.934032 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:10.934775 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:10.935263 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:10.935268 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:10.935272 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:10.935559 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:10.936104 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:10.936113 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:10.936523 (+ 410us) client_negotiation.cc:769] Sending connection context
0925 10:55:10.936793 (+ 270us) client_negotiation.cc:240] Negotiation successful
0925 10:55:10.937405 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":389,"threads_started":1}
I20260925 10:55:10.959960 7227 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:45311
I20260925 10:55:10.959973 7362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:45311 every 8 connection(s)
I20260925 10:55:10.961527 7227 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:55:10.964672 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 7227
I20260925 10:55:10.964923 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:55:10.965592 31994 external_mini_cluster.cc:1428] Setting key 9d98b28ab183a8828756cbc14f827671
I20260925 10:55:10.966095 7363 heartbeater.cc:347] Connected to a master server at 127.31.62.190:45847
I20260925 10:55:10.966323 7363 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:10.967191 7363 heartbeater.cc:511] Master 127.31.62.190:45847 requested a full tablet report, sending...
I20260925 10:55:10.967437 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:10.945298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:60991 (local address 127.31.62.190:45847)
0925 10:55:10.945466 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:10.945471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:10.946064 (+ 593us) server_negotiation.cc:407] Connection header received
0925 10:55:10.946280 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:10.946286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:10.946360 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:10.946449 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:10.948159 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:10.948997 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:10.951510 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:10.951928 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:10.952042 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:10.954925 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:10.954932 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:10.954937 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:10.954984 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:10.963810 (+ 8826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:10.965053 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:10.965061 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:10.965065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:10.965243 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:10.965730 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:10.965736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:10.965738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:10.966094 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:10.966259 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:10.966878 (+ 619us) server_negotiation.cc:299] Negotiation successful
0925 10:55:10.967092 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 10:55:10.967738 7323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:10.945350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:45847 (local address 127.31.62.129:60991)
0925 10:55:10.945934 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:10.945953 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:10.946133 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:10.946521 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:10.946526 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:10.947325 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:10.947951 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:10.947971 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:10.949154 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:10.949161 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:10.951267 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:10.951295 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:10.952157 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:10.952169 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:10.952380 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:10.953180 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:10.953216 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:10.954650 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:10.964010 (+ 9360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:10.964018 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:10.964024 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:10.964830 (+ 806us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:10.965404 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:10.965408 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:10.965412 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:10.965546 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:10.966363 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:10.966399 (+ 36us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:10.966624 (+ 225us) client_negotiation.cc:769] Sending connection context
0925 10:55:10.966753 (+ 129us) client_negotiation.cc:240] Negotiation successful
0925 10:55:10.967111 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":303,"threads_started":1}
I20260925 10:55:10.969422 7155 ts_manager.cc:194] Registered new tserver with Master: 5b669bbc2aa449b7baf228a3a5c50f6d (127.31.62.129:45311)
I20260925 10:55:10.970968 7155 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:35501
WARNING: no policy specified for oryx/127.31.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.130.keytab.
Entry for principal oryx/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.130.keytab.
I20260925 10:55:11.012632 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:45847
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.130.keytab
--principal=oryx/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:46743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:55:11.335680 7367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:11.335983 7367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:11.336109 7367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:11.343631 7367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:11.343760 7367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:11.343886 7367 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 10:55:11.343997 7367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:55:11.354342 7367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:46743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.130.keytab
--principal=oryx/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.62.190:45847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:11.355645 7367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:11.357124 7367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:11.370631 7372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:11.371031 7373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:11.371280 7375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:11.372735 7367 server_base.cc:1034] running on GCE node
I20260925 10:55:11.373416 7367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:11.374380 7367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:11.375629 7367 hybrid_clock.cc:648] HybridClock initialized: now 1790333711375510 us; error 65 us; skew 500 ppm
I20260925 10:55:11.375957 7367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:11 dist-test-slave-nx7s krb5kdc[7110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333711, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:11.379199 7367 init.cc:377] Logged in from keytab as oryx/127.31.62.130@KRBTEST.COM (short username oryx)
I20260925 10:55:11.380651 7367 webserver.cc:460] Webserver started at http://127.31.62.130:41679/ using document root <none> and password file <none>
I20260925 10:55:11.381408 7367 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:11.381496 7367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:11.381781 7367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:11.384694 7367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "a515c5d720cc439aba7d97e1ff90d1e0"
format_stamp: "Formatted at 2026-09-25 10:55:11 on dist-test-slave-nx7s"
server_key: "8bf0818b81734a8ab83c8abf135fa780"
server_key_iv: "45bafedcae5ee3da147185452e8b627e"
server_key_version: "encryptionkey@0"
I20260925 10:55:11.385565 7367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "a515c5d720cc439aba7d97e1ff90d1e0"
format_stamp: "Formatted at 2026-09-25 10:55:11 on dist-test-slave-nx7s"
server_key: "8bf0818b81734a8ab83c8abf135fa780"
server_key_iv: "45bafedcae5ee3da147185452e8b627e"
server_key_version: "encryptionkey@0"
I20260925 10:55:11.390879 7367 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260925 10:55:11.394670 7382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:11.396167 7367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:55:11.396318 7367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "a515c5d720cc439aba7d97e1ff90d1e0"
format_stamp: "Formatted at 2026-09-25 10:55:11 on dist-test-slave-nx7s"
server_key: "8bf0818b81734a8ab83c8abf135fa780"
server_key_iv: "45bafedcae5ee3da147185452e8b627e"
server_key_version: "encryptionkey@0"
I20260925 10:55:11.396529 7367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:11.407856 7367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:11.412014 7367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:11.412277 7367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:11.414916 7367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:11.415004 7367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:11.415131 7367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:11.415196 7367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 10:55:11 dist-test-slave-nx7s krb5kdc[7110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333711, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
I20260925 10:55:11.438192 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:11.415448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:55437 (local address 127.31.62.190:45847)
0925 10:55:11.415642 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:11.415648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:11.417101 (+ 1453us) server_negotiation.cc:407] Connection header received
0925 10:55:11.419183 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:11.419189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:11.419284 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:11.419388 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:11.422037 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.422830 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.425726 (+ 2896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.426171 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.426276 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:11.431325 (+ 5049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:11.431330 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:11.431333 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:11.431369 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:11.434486 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:11.435511 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:11.435518 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:11.435522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:11.435678 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:11.436145 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:11.436151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:11.436155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:11.436482 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:11.436632 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:11.437596 (+ 964us) server_negotiation.cc:299] Negotiation successful
0925 10:55:11.437844 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 10:55:11.439287 7391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:11.416063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:45847 (local address 127.31.62.130:55437)
0925 10:55:11.416910 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:11.416962 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:11.418850 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:11.419714 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:11.419745 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:11.420700 (+ 955us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:11.421808 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:11.421836 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.422983 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.422989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:11.425546 (+ 2557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:11.425567 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.426426 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.426434 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:11.426765 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:11.427655 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:11.427689 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:11.430992 (+ 3303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:11.434639 (+ 3647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:11.434652 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:11.434671 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:11.435327 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:11.435793 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:11.435799 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:11.435803 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:11.436014 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:11.436615 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:11.436624 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:11.437169 (+ 545us) client_negotiation.cc:769] Sending connection context
0925 10:55:11.437486 (+ 317us) client_negotiation.cc:240] Negotiation successful
0925 10:55:11.438076 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":325,"threads_started":1}
I20260925 10:55:11.459910 7454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:11.445361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:45847 (local address 127.31.62.130:58011)
0925 10:55:11.445975 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:11.445995 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:11.446134 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:11.446533 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:11.446537 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:11.447042 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:11.447343 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:11.447353 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.448910 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.448916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:11.450465 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:11.450482 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.451535 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.451543 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:11.451684 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:11.452314 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:11.452332 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:11.453376 (+ 1044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:11.457097 (+ 3721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:11.457103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:11.457107 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:11.457551 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:11.457914 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:11.457919 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:11.457923 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:11.458096 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:11.458912 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:11.458916 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:11.459097 (+ 181us) client_negotiation.cc:769] Sending connection context
0925 10:55:11.459224 (+ 127us) client_negotiation.cc:240] Negotiation successful
0925 10:55:11.459482 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":321,"threads_started":1}
I20260925 10:55:11.459944 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:11.445260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:58011 (local address 127.31.62.190:45847)
0925 10:55:11.445504 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:11.445511 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:11.446088 (+ 577us) server_negotiation.cc:407] Connection header received
0925 10:55:11.446247 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:11.446253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:11.446342 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:11.446472 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:11.447492 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.448678 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.450669 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.451314 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.451460 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:11.453657 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:11.453663 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:11.453668 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:11.453716 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:11.456959 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:11.457669 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:11.457675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:11.457679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:11.457805 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:11.458258 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:11.458264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:11.458269 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:11.458801 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:11.458961 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:11.459335 (+ 374us) server_negotiation.cc:299] Negotiation successful
0925 10:55:11.459594 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 10:55:11.470458 7367 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:44157
I20260925 10:55:11.470487 7501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:44157 every 8 connection(s)
I20260925 10:55:11.471664 7367 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:55:11.473289 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 7367
I20260925 10:55:11.473485 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:55:11.473902 7502 heartbeater.cc:347] Connected to a master server at 127.31.62.190:45847
I20260925 10:55:11.473986 31994 external_mini_cluster.cc:1428] Setting key a1daaba1ab5960a09216a09539758daa
I20260925 10:55:11.474112 7502 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:11.474838 7502 heartbeater.cc:511] Master 127.31.62.190:45847 requested a full tablet report, sending...
I20260925 10:55:11.476301 7155 ts_manager.cc:194] Registered new tserver with Master: a515c5d720cc439aba7d97e1ff90d1e0 (127.31.62.130:44157)
I20260925 10:55:11.477332 7155 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:55437
WARNING: no policy specified for oryx/127.31.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.131.keytab.
Entry for principal oryx/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.131.keytab.
I20260925 10:55:11.519791 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:45847
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.131.keytab
--principal=oryx/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:46743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:55:11.843070 7506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:11.843358 7506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:11.843451 7506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:11.850713 7506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:11.850831 7506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:11.850924 7506 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 10:55:11.851001 7506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:55:11.862643 7506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:46743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.131.keytab
--principal=oryx/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.62.190:45847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:11.864341 7506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:11.865981 7506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:11.879031 7511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:11.879078 7512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:11.879007 7514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:11.879700 7506 server_base.cc:1034] running on GCE node
I20260925 10:55:11.880345 7506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:11.881382 7506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:11.882617 7506 hybrid_clock.cc:648] HybridClock initialized: now 1790333711882533 us; error 57 us; skew 500 ppm
I20260925 10:55:11.882929 7506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:11 dist-test-slave-nx7s krb5kdc[7110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333711, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:11.886205 7506 init.cc:377] Logged in from keytab as oryx/127.31.62.131@KRBTEST.COM (short username oryx)
I20260925 10:55:11.887733 7506 webserver.cc:460] Webserver started at http://127.31.62.131:39901/ using document root <none> and password file <none>
I20260925 10:55:11.888440 7506 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:11.888526 7506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:11.888907 7506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:11.891943 7506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "26571f480c0048dc80063f243ebdcdb6"
format_stamp: "Formatted at 2026-09-25 10:55:11 on dist-test-slave-nx7s"
server_key: "d180f351e228369ba8854d9f7e51875e"
server_key_iv: "77a65844d829a2d297c21b1259a5f300"
server_key_version: "encryptionkey@0"
I20260925 10:55:11.892720 7506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "26571f480c0048dc80063f243ebdcdb6"
format_stamp: "Formatted at 2026-09-25 10:55:11 on dist-test-slave-nx7s"
server_key: "d180f351e228369ba8854d9f7e51875e"
server_key_iv: "77a65844d829a2d297c21b1259a5f300"
server_key_version: "encryptionkey@0"
I20260925 10:55:11.897890 7506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:55:11.901858 7521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:11.903390 7506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:55:11.903554 7506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "26571f480c0048dc80063f243ebdcdb6"
format_stamp: "Formatted at 2026-09-25 10:55:11 on dist-test-slave-nx7s"
server_key: "d180f351e228369ba8854d9f7e51875e"
server_key_iv: "77a65844d829a2d297c21b1259a5f300"
server_key_version: "encryptionkey@0"
I20260925 10:55:11.903767 7506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:11.927953 7506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:11.932300 7506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:11.932581 7506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:11.935283 7506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:11.935382 7506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:11.935497 7506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:11.935559 7506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 10:55:11 dist-test-slave-nx7s krb5kdc[7110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333711, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
I20260925 10:55:11.963306 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:11.936142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:52933 (local address 127.31.62.190:45847)
0925 10:55:11.936334 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:11.936340 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:11.937920 (+ 1580us) server_negotiation.cc:407] Connection header received
0925 10:55:11.941061 (+ 3141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:11.941067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:11.941162 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:11.941273 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:11.943693 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.944840 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.947276 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.947845 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.947988 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:11.953376 (+ 5388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:11.953384 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:11.953389 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:11.953429 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:11.956823 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:11.958078 (+ 1255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:11.958084 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:11.958088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:11.958284 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:11.958942 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:11.958947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:11.958951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:11.961240 (+ 2289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:11.961420 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:11.962683 (+ 1263us) server_negotiation.cc:299] Negotiation successful
0925 10:55:11.962933 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 10:55:11.964875 7530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:11.936742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:45847 (local address 127.31.62.131:52933)
0925 10:55:11.937699 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:11.937763 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:11.940593 (+ 2830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:11.941641 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:11.941655 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:11.942507 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:11.943485 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:11.943510 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.945087 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.945093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:11.947110 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:11.947126 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.948136 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.948144 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:11.948495 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:11.949343 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:11.949382 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:11.953045 (+ 3663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:11.957055 (+ 4010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:11.957068 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:11.957096 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:11.957920 (+ 824us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:11.958423 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:11.958432 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:11.958436 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:11.958768 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:11.961512 (+ 2744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:11.961526 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:11.962154 (+ 628us) client_negotiation.cc:769] Sending connection context
0925 10:55:11.962525 (+ 371us) client_negotiation.cc:240] Negotiation successful
0925 10:55:11.963148 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":337,"threads_started":1}
I20260925 10:55:11.973300 7506 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:40519
I20260925 10:55:11.973421 7639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:40519 every 8 connection(s)
I20260925 10:55:11.974371 7363 heartbeater.cc:503] Master 127.31.62.190:45847 was elected leader, sending a full tablet report...
I20260925 10:55:11.974701 7506 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:55:11.978525 7641 heartbeater.cc:347] Connected to a master server at 127.31.62.190:45847
I20260925 10:55:11.978875 7641 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:11.980011 7641 heartbeater.cc:511] Master 127.31.62.190:45847 requested a full tablet report, sending...
I20260925 10:55:11.981385 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 7506
I20260925 10:55:11.981585 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:55:11.982118 31994 external_mini_cluster.cc:1428] Setting key fbaad97bc8021cb182af67b5547bad74
I20260925 10:55:11.982151 7155 ts_manager.cc:194] Registered new tserver with Master: 26571f480c0048dc80063f243ebdcdb6 (127.31.62.131:40519)
I20260925 10:55:11.983505 7155 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:52933
I20260925 10:55:11.986038 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:55:11.998723 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:11.972718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:49387 (local address 127.31.62.190:45847)
0925 10:55:11.972958 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:11.972965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:11.985119 (+ 12154us) server_negotiation.cc:407] Connection header received
0925 10:55:11.985668 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:11.985675 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:11.985757 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:11.985831 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:11.987258 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.988056 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.989826 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.990449 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.990588 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:11.992558 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:11.992565 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:11.992569 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:11.992610 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:11.995861 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:11.996808 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:11.996812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:11.996815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:11.997008 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:11.997407 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:11.997411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:11.997414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:11.997685 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:11.997815 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:11.998303 (+ 488us) server_negotiation.cc:299] Negotiation successful
0925 10:55:11.998480 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 10:55:11.999070 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:11.972802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:45847 (local address 127.31.62.131:49387)
0925 10:55:11.984998 (+ 12196us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:11.985024 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:11.985483 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:11.986045 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:11.986051 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:11.986688 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:11.987067 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:11.987083 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.988231 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.988235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:11.989645 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:11.989664 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:11.990677 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:11.990682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:11.990825 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:11.991457 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:11.991477 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:11.992335 (+ 858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:11.996066 (+ 3731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:11.996073 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:11.996077 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:11.996647 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:11.997119 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:11.997124 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:11.997127 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:11.997289 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:11.997797 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:11.997802 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:11.998006 (+ 204us) client_negotiation.cc:769] Sending connection context
0925 10:55:11.998146 (+ 140us) client_negotiation.cc:240] Negotiation successful
0925 10:55:11.998453 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11984,"thread_start_us":12341,"threads_started":1}
Sep 25 10:55:12 dist-test-slave-nx7s krb5kdc[7110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333712, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 10:55:12 dist-test-slave-nx7s krb5kdc[7110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:12 dist-test-slave-nx7s krb5kdc[7110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:12.480099 7502 heartbeater.cc:503] Master 127.31.62.190:45847 was elected leader, sending a full tablet report...
I20260925 10:55:12.987759 7641 heartbeater.cc:503] Master 127.31.62.190:45847 was elected leader, sending a full tablet report...
W20260925 10:55:13.008738 7649 sasl_common.cc:323] Time spent calling sasl_client_start(): real 0.975s user 0.005s sys 0.008s
W20260925 10:55:13.009944 7251 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:12.028174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54332 (local address 127.31.62.190:45847)
0925 10:55:12.028361 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:12.028365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:12.028735 (+ 370us) server_negotiation.cc:407] Connection header received
0925 10:55:12.028937 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:12.028941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:12.029005 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:12.029096 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:12.030390 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:12.031111 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:12.032323 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:12.032746 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:12.032886 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.009666 (+976780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54332: BlockingRecv error: recv got EOF from 127.0.0.1:54332 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 25 10:55:13 dist-test-slave-nx7s krb5kdc[7110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333712, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
I20260925 10:55:13.028137 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.013904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54344 (local address 127.31.62.190:45847)
0925 10:55:13.014063 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:13.014068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:13.014361 (+ 293us) server_negotiation.cc:407] Connection header received
0925 10:55:13.014532 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:13.014536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:13.014624 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:13.014703 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:13.016004 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.016764 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.018289 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.018751 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.018872 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.022887 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:13.022892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:13.022896 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:13.022936 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:13.025493 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:13.026213 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:13.026217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:13.026220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:13.026358 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:13.026750 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:13.026754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:13.026756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:13.027055 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:13.027299 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:13.027667 (+ 368us) server_negotiation.cc:299] Negotiation successful
0925 10:55:13.027883 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 10:55:13.032758 7155 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:55:13.039790 7155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:55:13.074735 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.055940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60272 (local address 127.31.62.131:40519)
0925 10:55:13.063194 (+ 7254us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:13.063206 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:13.067765 (+ 4559us) server_negotiation.cc:407] Connection header received
0925 10:55:13.067940 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:13.067946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:13.068138 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:13.068299 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:13.068816 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.070053 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.072254 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.073360 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.073480 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.073694 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:13.074265 (+ 571us) server_negotiation.cc:299] Negotiation successful
0925 10:55:13.074432 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7006,"thread_start_us":7111,"threads_started":1}
I20260925 10:55:13.074805 7664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.055372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:40519 (local address 127.0.0.1:60272)
0925 10:55:13.067638 (+ 12266us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:13.067652 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:13.067792 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:13.068310 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:13.068314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:13.068335 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:13.068652 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.068664 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.070223 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.070227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.072100 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.072111 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.073823 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.073829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.074010 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.074085 (+ 75us) client_negotiation.cc:769] Sending connection context
0925 10:55:13.074209 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 10:55:13.074414 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12157,"mutex_wait_us":54,"spinlock_wait_cycles":896,"thread_start_us":12032,"threads_started":1}
I20260925 10:55:13.075661 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.053075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47354 (local address 127.31.62.130:44157)
0925 10:55:13.065811 (+ 12736us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:13.065820 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:13.066387 (+ 567us) server_negotiation.cc:407] Connection header received
0925 10:55:13.066823 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:13.066832 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:13.067100 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:13.067335 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:13.068481 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.070011 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.072974 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.074382 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.074486 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.074683 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:13.075212 (+ 529us) server_negotiation.cc:299] Negotiation successful
0925 10:55:13.075383 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12580,"thread_start_us":12754,"threads_started":1}
I20260925 10:55:13.075762 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.052938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:44157 (local address 127.0.0.1:47354)
0925 10:55:13.066277 (+ 13339us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:13.066294 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:13.066470 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:13.067634 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:13.067650 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:13.067690 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:13.068229 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.068237 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.070201 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.070206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.072795 (+ 2589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.072805 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.074877 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.074882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.075030 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.075061 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 10:55:13.075152 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 10:55:13.075278 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13212,"mutex_wait_us":206,"spinlock_wait_cycles":5376,"thread_start_us":12954,"threads_started":1}
I20260925 10:55:13.075922 7660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.052927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:45311 (local address 127.0.0.1:53118)
0925 10:55:13.065493 (+ 12566us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:13.066103 (+ 610us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:13.066437 (+ 334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:13.067634 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:13.067642 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:13.067684 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:13.068200 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.068226 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.070201 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.070206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.072769 (+ 2563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.072788 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.075161 (+ 2373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.075167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.075322 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.075352 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 10:55:13.075431 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 10:55:13.075557 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12253,"mutex_wait_us":8,"spinlock_wait_cycles":1408,"thread_start_us":12062,"threads_started":1}
I20260925 10:55:13.076581 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.053484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53118 (local address 127.31.62.129:45311)
0925 10:55:13.065240 (+ 11756us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:13.065251 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:13.066254 (+ 1003us) server_negotiation.cc:407] Connection header received
0925 10:55:13.066823 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:13.066832 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:13.067100 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:13.067336 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:13.068438 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.070010 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.073172 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.074554 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.074646 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.074830 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:13.076152 (+ 1322us) server_negotiation.cc:299] Negotiation successful
0925 10:55:13.076316 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11568,"thread_start_us":11755,"threads_started":1}
I20260925 10:55:13.079255 7432 tablet_service.cc:1467] Processing CreateTablet for tablet e6ec1cb3309042dea344893efb95d5c1 (DEFAULT_TABLE table=test-table [id=988fde72bee047afb04e11ac56b2d32c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:13.079248 7571 tablet_service.cc:1467] Processing CreateTablet for tablet e6ec1cb3309042dea344893efb95d5c1 (DEFAULT_TABLE table=test-table [id=988fde72bee047afb04e11ac56b2d32c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:13.079726 7293 tablet_service.cc:1467] Processing CreateTablet for tablet e6ec1cb3309042dea344893efb95d5c1 (DEFAULT_TABLE table=test-table [id=988fde72bee047afb04e11ac56b2d32c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:13.080641 7571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6ec1cb3309042dea344893efb95d5c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:13.080639 7432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6ec1cb3309042dea344893efb95d5c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:13.080822 7293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6ec1cb3309042dea344893efb95d5c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:13.093557 7666 tablet_bootstrap.cc:492] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d: Bootstrap starting.
I20260925 10:55:13.094946 7667 tablet_bootstrap.cc:492] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6: Bootstrap starting.
I20260925 10:55:13.097731 7666 tablet_bootstrap.cc:654] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:13.098265 7667 tablet_bootstrap.cc:654] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:13.098994 7666 log.cc:824] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:13.099431 7667 log.cc:824] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:13.101131 7668 tablet_bootstrap.cc:492] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0: Bootstrap starting.
I20260925 10:55:13.102427 7666 tablet_bootstrap.cc:492] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d: No bootstrap required, opened a new log
I20260925 10:55:13.102643 7667 tablet_bootstrap.cc:492] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6: No bootstrap required, opened a new log
I20260925 10:55:13.102717 7666 ts_tablet_manager.cc:1397] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260925 10:55:13.102901 7667 ts_tablet_manager.cc:1397] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260925 10:55:13.104557 7668 tablet_bootstrap.cc:654] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:13.105893 7668 log.cc:824] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:13.109071 7668 tablet_bootstrap.cc:492] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0: No bootstrap required, opened a new log
I20260925 10:55:13.109369 7668 ts_tablet_manager.cc:1397] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260925 10:55:13.111024 7666 raft_consensus.cc:348] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.111397 7666 raft_consensus.cc:374] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:13.111512 7666 raft_consensus.cc:729] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b669bbc2aa449b7baf228a3a5c50f6d, State: Initialized, Role: FOLLOWER
I20260925 10:55:13.112115 7666 consensus_queue.cc:260] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.112172 7667 raft_consensus.cc:348] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.112653 7667 raft_consensus.cc:374] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:13.112788 7667 raft_consensus.cc:729] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26571f480c0048dc80063f243ebdcdb6, State: Initialized, Role: FOLLOWER
I20260925 10:55:13.113653 7666 ts_tablet_manager.cc:1428] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260925 10:55:13.114399 7667 consensus_queue.cc:260] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.116042 7667 ts_tablet_manager.cc:1428] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260925 10:55:13.119227 7668 raft_consensus.cc:348] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.119679 7668 raft_consensus.cc:374] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:13.119781 7668 raft_consensus.cc:729] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a515c5d720cc439aba7d97e1ff90d1e0, State: Initialized, Role: FOLLOWER
I20260925 10:55:13.120309 7668 consensus_queue.cc:260] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.121887 7668 ts_tablet_manager.cc:1428] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260925 10:55:13.170477 7672 raft_consensus.cc:482] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:55:13.170769 7672 raft_consensus.cc:504] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.172361 7672 leader_election.cc:288] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a515c5d720cc439aba7d97e1ff90d1e0 (127.31.62.130:44157), 26571f480c0048dc80063f243ebdcdb6 (127.31.62.131:40519)
I20260925 10:55:13.179320 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.172922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:47173 (local address 127.31.62.131:40519)
0925 10:55:13.173115 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:13.173122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:13.173391 (+ 269us) server_negotiation.cc:407] Connection header received
0925 10:55:13.173631 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:13.173636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:13.173711 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:13.173813 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:13.174384 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.175292 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.177375 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.178317 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.178442 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.178498 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:13.178934 (+ 436us) server_negotiation.cc:299] Negotiation successful
0925 10:55:13.179071 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 10:55:13.179441 7676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.172638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:40519 (local address 127.31.62.129:47173)
0925 10:55:13.173272 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:13.173289 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:13.173461 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:13.173832 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:13.173839 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:13.173861 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:13.174234 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.174247 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.175411 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.175416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.177233 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.177244 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.178530 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.178537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.178736 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.178794 (+ 58us) client_negotiation.cc:769] Sending connection context
0925 10:55:13.178896 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 10:55:13.179037 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":3,"thread_start_us":476,"threads_started":1}
I20260925 10:55:13.179737 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.172892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:35643 (local address 127.31.62.130:44157)
0925 10:55:13.173076 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:13.173081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:13.173967 (+ 886us) server_negotiation.cc:407] Connection header received
0925 10:55:13.174197 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:13.174203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:13.174275 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:13.174363 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:13.174931 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.175846 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.177841 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.178821 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.178963 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.179028 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:13.179400 (+ 372us) server_negotiation.cc:299] Negotiation successful
0925 10:55:13.179532 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 10:55:13.179867 7675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.172638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:44157 (local address 127.31.62.129:35643)
0925 10:55:13.173853 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:13.173871 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:13.174005 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:13.174384 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:13.174389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:13.174411 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:13.174737 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.174791 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.175986 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.175991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.177603 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.177617 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.179018 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.179025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.179220 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.179250 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 10:55:13.179375 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 10:55:13.179509 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1067,"mutex_wait_us":53,"thread_start_us":841,"threads_started":1}
I20260925 10:55:13.180596 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6ec1cb3309042dea344893efb95d5c1" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26571f480c0048dc80063f243ebdcdb6" is_pre_election: true
I20260925 10:55:13.180790 7457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6ec1cb3309042dea344893efb95d5c1" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" is_pre_election: true
I20260925 10:55:13.181093 7591 raft_consensus.cc:2454] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 0.
I20260925 10:55:13.181294 7457 raft_consensus.cc:2454] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 0.
I20260925 10:55:13.181743 7246 leader_election.cc:302] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26571f480c0048dc80063f243ebdcdb6, 5b669bbc2aa449b7baf228a3a5c50f6d; no voters:
I20260925 10:55:13.182241 7672 raft_consensus.cc:2781] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:55:13.182375 7672 raft_consensus.cc:482] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:55:13.182471 7672 raft_consensus.cc:3037] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:13.184612 7672 raft_consensus.cc:504] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.185289 7672 leader_election.cc:288] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 election: Requested vote from peers a515c5d720cc439aba7d97e1ff90d1e0 (127.31.62.130:44157), 26571f480c0048dc80063f243ebdcdb6 (127.31.62.131:40519)
I20260925 10:55:13.185838 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6ec1cb3309042dea344893efb95d5c1" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26571f480c0048dc80063f243ebdcdb6"
I20260925 10:55:13.185846 7457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6ec1cb3309042dea344893efb95d5c1" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a515c5d720cc439aba7d97e1ff90d1e0"
I20260925 10:55:13.186023 7457 raft_consensus.cc:3037] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:13.186026 7591 raft_consensus.cc:3037] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:13.188154 7457 raft_consensus.cc:2454] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 1.
I20260925 10:55:13.188186 7591 raft_consensus.cc:2454] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 1.
I20260925 10:55:13.188620 7245 leader_election.cc:302] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b669bbc2aa449b7baf228a3a5c50f6d, a515c5d720cc439aba7d97e1ff90d1e0; no voters:
I20260925 10:55:13.189002 7672 raft_consensus.cc:2781] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:55:13.189488 7672 raft_consensus.cc:686] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 LEADER]: Becoming Leader. State: Replica: 5b669bbc2aa449b7baf228a3a5c50f6d, State: Running, Role: LEADER
I20260925 10:55:13.190034 7672 consensus_queue.cc:237] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.195078 7155 catalog_manager.cc:5495] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b669bbc2aa449b7baf228a3a5c50f6d (127.31.62.129). New cstate: current_term: 1 leader_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } health_report { overall_health: UNKNOWN } } }
W20260925 10:55:13.216984 7364 tablet.cc:2369] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:55:13.224455 7503 tablet.cc:2369] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:55:13.232987 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.227413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53130 (local address 127.31.62.129:45311)
0925 10:55:13.227609 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:13.227615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:13.227634 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:55:13.227878 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:13.227883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:13.227988 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:13.228209 (+ 221us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:55:13.228580 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.229409 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.230782 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.231175 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.231273 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.231753 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:55:13.232322 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:55:13.232441 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:13.232588 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 10:55:13.232679 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260925 10:55:13.241080 7642 tablet.cc:2369] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:55:13.241685 7457 raft_consensus.cc:1261] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Refusing update from remote peer 5b669bbc2aa449b7baf228a3a5c50f6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:55:13.241968 7591 raft_consensus.cc:1261] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Refusing update from remote peer 5b669bbc2aa449b7baf228a3a5c50f6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:55:13.242837 7677 consensus_queue.cc:1035] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:13.243199 7672 consensus_queue.cc:1035] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:13.258002 7681 mvcc.cc:205] Tried to move back new op lower bound from 7333206889426178048 to 7333206889230344192. Current Snapshot: MvccSnapshot[applied={T|T < 7333206889426178048}]
I20260925 10:55:13.270244 7155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:55:13.270495 7155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:55:13.274638 7155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927: Sending DeleteTablet for 3 replicas of tablet e6ec1cb3309042dea344893efb95d5c1
I20260925 10:55:13.276440 7432 tablet_service.cc:1514] Processing DeleteTablet for tablet e6ec1cb3309042dea344893efb95d5c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:13 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:47354
I20260925 10:55:13.278174 7690 tablet_replica.cc:330] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0: stopping tablet replica
I20260925 10:55:13.278714 7690 raft_consensus.cc:2229] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:13.279304 7432 tablet_service.cc:1514] Processing DeleteTablet for tablet e6ec1cb3309042dea344893efb95d5c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:13 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:47354
I20260925 10:55:13.279309 7690 raft_consensus.cc:2258] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:13.279627 7293 tablet_service.cc:1514] Processing DeleteTablet for tablet e6ec1cb3309042dea344893efb95d5c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:13 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:53118
I20260925 10:55:13.280547 7691 tablet_replica.cc:330] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d: stopping tablet replica
I20260925 10:55:13.281015 7691 raft_consensus.cc:2229] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:55:13.281054 7571 tablet_service.cc:1514] Processing DeleteTablet for tablet e6ec1cb3309042dea344893efb95d5c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:13 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:60272
I20260925 10:55:13.281409 7690 ts_tablet_manager.cc:1905] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:13.281667 7691 raft_consensus.cc:2258] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:13.283663 7691 ts_tablet_manager.cc:1905] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:13.285441 7154 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:55:13.286295 7154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:55:13.290318 7690 ts_tablet_manager.cc:1918] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:55:13.290606 7690 log.cc:1196] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/e6ec1cb3309042dea344893efb95d5c1
I20260925 10:55:13.291224 7690 ts_tablet_manager.cc:1939] T e6ec1cb3309042dea344893efb95d5c1 P a515c5d720cc439aba7d97e1ff90d1e0: Deleting consensus metadata
I20260925 10:55:13.293509 7141 catalog_manager.cc:4841] TS a515c5d720cc439aba7d97e1ff90d1e0 (127.31.62.130:44157): tablet e6ec1cb3309042dea344893efb95d5c1 (table test-table [id=988fde72bee047afb04e11ac56b2d32c]) successfully deleted
I20260925 10:55:13.293874 7691 ts_tablet_manager.cc:1918] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:55:13.294087 7691 log.cc:1196] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/e6ec1cb3309042dea344893efb95d5c1
I20260925 10:55:13.294680 7691 ts_tablet_manager.cc:1939] T e6ec1cb3309042dea344893efb95d5c1 P 5b669bbc2aa449b7baf228a3a5c50f6d: Deleting consensus metadata
W20260925 10:55:13.295300 7141 catalog_manager.cc:4808] TS a515c5d720cc439aba7d97e1ff90d1e0 (127.31.62.130:44157): delete failed for tablet e6ec1cb3309042dea344893efb95d5c1 because the tablet was not found. No further retry: Not found: Tablet not found: e6ec1cb3309042dea344893efb95d5c1
I20260925 10:55:13.296204 7570 tablet_service.cc:1467] Processing CreateTablet for tablet 1fdbbf6df527453d985df21dabf4b7a0 (DEFAULT_TABLE table=test-table [id=b3c0f08f36f745bab4f4cbb75f1e7225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:13.296178 7432 tablet_service.cc:1467] Processing CreateTablet for tablet 1fdbbf6df527453d985df21dabf4b7a0 (DEFAULT_TABLE table=test-table [id=b3c0f08f36f745bab4f4cbb75f1e7225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:13.296617 7432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fdbbf6df527453d985df21dabf4b7a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:13.296707 7570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fdbbf6df527453d985df21dabf4b7a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:13.299954 7139 catalog_manager.cc:4841] TS 5b669bbc2aa449b7baf228a3a5c50f6d (127.31.62.129:45311): tablet e6ec1cb3309042dea344893efb95d5c1 (table test-table [id=988fde72bee047afb04e11ac56b2d32c]) successfully deleted
I20260925 10:55:13.300406 7293 tablet_service.cc:1467] Processing CreateTablet for tablet 1fdbbf6df527453d985df21dabf4b7a0 (DEFAULT_TABLE table=test-table [id=b3c0f08f36f745bab4f4cbb75f1e7225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:13.300944 7293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fdbbf6df527453d985df21dabf4b7a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:13.305011 7668 tablet_bootstrap.cc:492] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0: Bootstrap starting.
I20260925 10:55:13.306594 7668 tablet_bootstrap.cc:654] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:13.308434 7667 tablet_bootstrap.cc:492] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6: Bootstrap starting.
I20260925 10:55:13.308940 7668 tablet_bootstrap.cc:492] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0: No bootstrap required, opened a new log
I20260925 10:55:13.309324 7668 ts_tablet_manager.cc:1397] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 10:55:13.310063 7667 tablet_bootstrap.cc:654] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:13.310125 7668 raft_consensus.cc:348] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.310344 7668 raft_consensus.cc:374] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:13.310432 7668 raft_consensus.cc:729] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a515c5d720cc439aba7d97e1ff90d1e0, State: Initialized, Role: FOLLOWER
I20260925 10:55:13.310528 7666 tablet_bootstrap.cc:492] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d: Bootstrap starting.
I20260925 10:55:13.310659 7668 consensus_queue.cc:260] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.311213 7668 ts_tablet_manager.cc:1428] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 10:55:13.312064 7666 tablet_bootstrap.cc:654] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:13.315004 7666 tablet_bootstrap.cc:492] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d: No bootstrap required, opened a new log
I20260925 10:55:13.315145 7666 ts_tablet_manager.cc:1397] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 10:55:13.316012 7666 raft_consensus.cc:348] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.316233 7666 raft_consensus.cc:374] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:13.316329 7666 raft_consensus.cc:729] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b669bbc2aa449b7baf228a3a5c50f6d, State: Initialized, Role: FOLLOWER
I20260925 10:55:13.316741 7666 consensus_queue.cc:260] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.317899 7666 ts_tablet_manager.cc:1428] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 10:55:13.319542 7692 tablet_replica.cc:330] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6: stopping tablet replica
I20260925 10:55:13.320071 7692 raft_consensus.cc:2229] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:13.320418 7667 tablet_bootstrap.cc:492] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6: No bootstrap required, opened a new log
I20260925 10:55:13.320572 7667 ts_tablet_manager.cc:1397] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260925 10:55:13.320770 7692 raft_consensus.cc:2258] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:13.321427 7667 raft_consensus.cc:348] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.321627 7667 raft_consensus.cc:374] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:13.321731 7667 raft_consensus.cc:729] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26571f480c0048dc80063f243ebdcdb6, State: Initialized, Role: FOLLOWER
I20260925 10:55:13.321964 7667 consensus_queue.cc:260] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.322420 7667 ts_tablet_manager.cc:1428] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 10:55:13.323040 7692 ts_tablet_manager.cc:1905] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:13.330256 7692 ts_tablet_manager.cc:1918] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:55:13.330451 7692 log.cc:1196] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/e6ec1cb3309042dea344893efb95d5c1
I20260925 10:55:13.331013 7692 ts_tablet_manager.cc:1939] T e6ec1cb3309042dea344893efb95d5c1 P 26571f480c0048dc80063f243ebdcdb6: Deleting consensus metadata
I20260925 10:55:13.332892 7142 catalog_manager.cc:4841] TS 26571f480c0048dc80063f243ebdcdb6 (127.31.62.131:40519): tablet e6ec1cb3309042dea344893efb95d5c1 (table test-table [id=988fde72bee047afb04e11ac56b2d32c]) successfully deleted
I20260925 10:55:13.333847 7677 raft_consensus.cc:482] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:55:13.334057 7677 raft_consensus.cc:504] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.334777 7677 leader_election.cc:288] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a515c5d720cc439aba7d97e1ff90d1e0 (127.31.62.130:44157), 26571f480c0048dc80063f243ebdcdb6 (127.31.62.131:40519)
I20260925 10:55:13.335239 7457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fdbbf6df527453d985df21dabf4b7a0" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" is_pre_election: true
I20260925 10:55:13.335487 7457 raft_consensus.cc:2454] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 0.
I20260925 10:55:13.335464 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fdbbf6df527453d985df21dabf4b7a0" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26571f480c0048dc80063f243ebdcdb6" is_pre_election: true
I20260925 10:55:13.335675 7591 raft_consensus.cc:2454] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 0.
I20260925 10:55:13.335963 7245 leader_election.cc:302] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b669bbc2aa449b7baf228a3a5c50f6d, a515c5d720cc439aba7d97e1ff90d1e0; no voters:
I20260925 10:55:13.336241 7677 raft_consensus.cc:2781] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:55:13.336344 7677 raft_consensus.cc:482] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:55:13.336438 7677 raft_consensus.cc:3037] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:13.339637 7677 raft_consensus.cc:504] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.340687 7457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fdbbf6df527453d985df21dabf4b7a0" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a515c5d720cc439aba7d97e1ff90d1e0"
I20260925 10:55:13.340690 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fdbbf6df527453d985df21dabf4b7a0" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26571f480c0048dc80063f243ebdcdb6"
I20260925 10:55:13.340914 7457 raft_consensus.cc:3037] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:13.340946 7591 raft_consensus.cc:3037] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:13.343777 7457 raft_consensus.cc:2454] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 1.
I20260925 10:55:13.343693 7591 raft_consensus.cc:2454] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 1.
I20260925 10:55:13.344322 7245 leader_election.cc:302] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b669bbc2aa449b7baf228a3a5c50f6d, a515c5d720cc439aba7d97e1ff90d1e0; no voters:
I20260925 10:55:13.344498 7677 leader_election.cc:288] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 election: Requested vote from peers a515c5d720cc439aba7d97e1ff90d1e0 (127.31.62.130:44157), 26571f480c0048dc80063f243ebdcdb6 (127.31.62.131:40519)
I20260925 10:55:13.344619 7672 raft_consensus.cc:2781] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:55:13.344868 7672 raft_consensus.cc:686] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 LEADER]: Becoming Leader. State: Replica: 5b669bbc2aa449b7baf228a3a5c50f6d, State: Running, Role: LEADER
I20260925 10:55:13.345152 7672 consensus_queue.cc:237] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.350560 7154 catalog_manager.cc:5495] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b669bbc2aa449b7baf228a3a5c50f6d (127.31.62.129). New cstate: current_term: 1 leader_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } health_report { overall_health: UNKNOWN } } }
Sep 25 10:55:13 dist-test-slave-nx7s krb5kdc[7110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333710, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.190@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
I20260925 10:55:13.398343 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.383307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54346 (local address 127.31.62.190:45847)
0925 10:55:13.383552 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:13.383562 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:13.384064 (+ 502us) server_negotiation.cc:407] Connection header received
0925 10:55:13.384232 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:13.384236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:13.384305 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:13.384390 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:13.385901 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.386620 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.388439 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.388933 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.389047 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.392952 (+ 3905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:13.392957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:13.392960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:13.392989 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:13.395421 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:13.396275 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:13.396280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:13.396282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:13.396395 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:13.396771 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:13.396774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:13.396776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:13.397061 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:13.397182 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:13.397776 (+ 594us) server_negotiation.cc:299] Negotiation successful
0925 10:55:13.398048 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":115}
I20260925 10:55:13.398384 7702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.383312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:45847 (local address 127.0.0.1:54346)
0925 10:55:13.383947 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:13.383967 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:13.384129 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:13.384439 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:13.384443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:13.385427 (+ 984us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:13.385758 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.385771 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.386742 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.386747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.388309 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.388325 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.389122 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.389128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.389306 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.390012 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:13.390034 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:13.392747 (+ 2713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:13.395568 (+ 2821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:13.395583 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:13.395591 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:13.396158 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:13.396492 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:13.396497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:13.396501 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:13.396665 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:13.397181 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:13.397190 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:13.397466 (+ 276us) client_negotiation.cc:769] Sending connection context
0925 10:55:13.397650 (+ 184us) client_negotiation.cc:240] Negotiation successful
0925 10:55:13.397925 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":325,"threads_started":1}
I20260925 10:55:13.403689 7154 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:54346:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 10:55:13.405056 7154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:55:13.412405 7293 tablet_service.cc:1467] Processing CreateTablet for tablet f170538f8d8947b8bef344a821a96ef5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d2df303c13254834a9ace6561f9c387e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 10:55:13.412688 7571 tablet_service.cc:1467] Processing CreateTablet for tablet f170538f8d8947b8bef344a821a96ef5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d2df303c13254834a9ace6561f9c387e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 10:55:13.412730 7432 tablet_service.cc:1467] Processing CreateTablet for tablet f170538f8d8947b8bef344a821a96ef5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d2df303c13254834a9ace6561f9c387e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 10:55:13.413074 7293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f170538f8d8947b8bef344a821a96ef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:13.413223 7571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f170538f8d8947b8bef344a821a96ef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:13.413247 7432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f170538f8d8947b8bef344a821a96ef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:13.420128 7668 tablet_bootstrap.cc:492] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0: Bootstrap starting.
I20260925 10:55:13.420145 7667 tablet_bootstrap.cc:492] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6: Bootstrap starting.
I20260925 10:55:13.421729 7667 tablet_bootstrap.cc:654] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:13.421787 7668 tablet_bootstrap.cc:654] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:13.423156 7666 tablet_bootstrap.cc:492] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d: Bootstrap starting.
I20260925 10:55:13.424575 7666 tablet_bootstrap.cc:654] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:13.426093 7667 tablet_bootstrap.cc:492] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6: No bootstrap required, opened a new log
I20260925 10:55:13.426256 7667 ts_tablet_manager.cc:1397] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260925 10:55:13.426827 7667 raft_consensus.cc:348] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.427017 7667 raft_consensus.cc:374] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:13.427091 7667 raft_consensus.cc:729] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26571f480c0048dc80063f243ebdcdb6, State: Initialized, Role: FOLLOWER
I20260925 10:55:13.427331 7667 consensus_queue.cc:260] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.427865 7673 tablet_replica.cc:439] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } } }
I20260925 10:55:13.428177 7673 tablet_replica.cc:442] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:55:13.429198 7667 ts_tablet_manager.cc:1428] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260925 10:55:13.432060 7666 tablet_bootstrap.cc:492] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d: No bootstrap required, opened a new log
I20260925 10:55:13.432111 7668 tablet_bootstrap.cc:492] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0: No bootstrap required, opened a new log
I20260925 10:55:13.432256 7666 ts_tablet_manager.cc:1397] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260925 10:55:13.432276 7668 ts_tablet_manager.cc:1397] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260925 10:55:13.433017 7666 raft_consensus.cc:348] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.433014 7668 raft_consensus.cc:348] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.433218 7666 raft_consensus.cc:374] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:13.433259 7668 raft_consensus.cc:374] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:13.433300 7666 raft_consensus.cc:729] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b669bbc2aa449b7baf228a3a5c50f6d, State: Initialized, Role: FOLLOWER
I20260925 10:55:13.433323 7668 raft_consensus.cc:729] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a515c5d720cc439aba7d97e1ff90d1e0, State: Initialized, Role: FOLLOWER
I20260925 10:55:13.433554 7666 consensus_queue.cc:260] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.433590 7668 consensus_queue.cc:260] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.434114 7666 ts_tablet_manager.cc:1428] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 10:55:13.434152 7668 ts_tablet_manager.cc:1428] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 10:55:13.434872 7674 tablet_replica.cc:439] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } } }
I20260925 10:55:13.435225 7674 tablet_replica.cc:442] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:55:13.433945 7672 tablet_replica.cc:439] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } } }
I20260925 10:55:13.435662 7672 tablet_replica.cc:442] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:55:13.460835 7672 raft_consensus.cc:482] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:55:13.461071 7672 raft_consensus.cc:504] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.461727 7672 leader_election.cc:288] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a515c5d720cc439aba7d97e1ff90d1e0 (127.31.62.130:44157), 26571f480c0048dc80063f243ebdcdb6 (127.31.62.131:40519)
I20260925 10:55:13.462273 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f170538f8d8947b8bef344a821a96ef5" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26571f480c0048dc80063f243ebdcdb6" is_pre_election: true
I20260925 10:55:13.462247 7457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f170538f8d8947b8bef344a821a96ef5" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" is_pre_election: true
I20260925 10:55:13.462534 7591 raft_consensus.cc:2454] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 0.
I20260925 10:55:13.462554 7457 raft_consensus.cc:2454] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 0.
I20260925 10:55:13.462952 7245 leader_election.cc:302] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b669bbc2aa449b7baf228a3a5c50f6d, a515c5d720cc439aba7d97e1ff90d1e0; no voters:
I20260925 10:55:13.463294 7672 raft_consensus.cc:2781] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:55:13.463402 7672 raft_consensus.cc:482] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:55:13.463508 7672 raft_consensus.cc:3037] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:13.466642 7672 raft_consensus.cc:504] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.467427 7672 leader_election.cc:288] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 election: Requested vote from peers a515c5d720cc439aba7d97e1ff90d1e0 (127.31.62.130:44157), 26571f480c0048dc80063f243ebdcdb6 (127.31.62.131:40519)
I20260925 10:55:13.467851 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f170538f8d8947b8bef344a821a96ef5" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26571f480c0048dc80063f243ebdcdb6"
I20260925 10:55:13.467883 7457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f170538f8d8947b8bef344a821a96ef5" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a515c5d720cc439aba7d97e1ff90d1e0"
I20260925 10:55:13.468089 7591 raft_consensus.cc:3037] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:13.468103 7457 raft_consensus.cc:3037] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:13.470198 7591 raft_consensus.cc:2454] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 1.
I20260925 10:55:13.470781 7246 leader_election.cc:302] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26571f480c0048dc80063f243ebdcdb6, 5b669bbc2aa449b7baf228a3a5c50f6d; no voters:
I20260925 10:55:13.471076 7457 raft_consensus.cc:2454] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 1.
I20260925 10:55:13.471120 7672 raft_consensus.cc:2781] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:55:13.471695 7672 raft_consensus.cc:686] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 LEADER]: Becoming Leader. State: Replica: 5b669bbc2aa449b7baf228a3a5c50f6d, State: Running, Role: LEADER
I20260925 10:55:13.472023 7672 consensus_queue.cc:237] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.477583 7677 tablet_replica.cc:439] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d: TxnStatusTablet state changed. Reason: New leader 5b669bbc2aa449b7baf228a3a5c50f6d. Latest consensus state: current_term: 1 leader_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } } }
I20260925 10:55:13.477888 7677 tablet_replica.cc:442] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d: This TxnStatusTablet replica's current role is: LEADER
I20260925 10:55:13.479781 7155 catalog_manager.cc:5495] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b669bbc2aa449b7baf228a3a5c50f6d (127.31.62.129). New cstate: current_term: 1 leader_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } health_report { overall_health: UNKNOWN } } }
I20260925 10:55:13.485270 7707 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 10:55:13.485533 7707 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 10:55:13.499184 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.492488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53134 (local address 127.31.62.129:45311)
0925 10:55:13.492669 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:13.492676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:13.493161 (+ 485us) server_negotiation.cc:407] Connection header received
0925 10:55:13.493413 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:13.493419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:13.493499 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:13.493596 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:55:13.493971 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.495149 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.497125 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.497796 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.497918 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.498410 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:55:13.498561 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:55:13.498649 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:13.498791 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 10:55:13.498874 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 10:55:13.499217 7702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.492427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:45311 (local address 127.0.0.1:53134)
0925 10:55:13.493057 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:13.493077 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:13.493233 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:13.493627 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:13.493632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:13.493645 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 10:55:13.493841 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.493851 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.495311 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.495317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.496976 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.496992 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.497956 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.497962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.498159 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.498237 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 10:55:13.498661 (+ 424us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 10:55:13.498681 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 10:55:13.498764 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 10:55:13.498919 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466}
I20260925 10:55:13.503167 7591 raft_consensus.cc:1261] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Refusing update from remote peer 5b669bbc2aa449b7baf228a3a5c50f6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:55:13.503257 7457 raft_consensus.cc:1261] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Refusing update from remote peer 5b669bbc2aa449b7baf228a3a5c50f6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:55:13.503870 7672 consensus_queue.cc:1035] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:13.504101 7677 consensus_queue.cc:1035] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:13.507452 7673 tablet_replica.cc:439] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6: TxnStatusTablet state changed. Reason: New leader 5b669bbc2aa449b7baf228a3a5c50f6d. Latest consensus state: current_term: 1 leader_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } } }
I20260925 10:55:13.507797 7673 tablet_replica.cc:442] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:55:13.509572 7687 tablet_replica.cc:439] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } } }
I20260925 10:55:13.509936 7687 tablet_replica.cc:442] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d: This TxnStatusTablet replica's current role is: LEADER
I20260925 10:55:13.510118 7680 mvcc.cc:205] Tried to move back new op lower bound from 7333206890502987776 to 7333206890384347136. Current Snapshot: MvccSnapshot[applied={T|T < 7333206890502987776}]
I20260925 10:55:13.511581 7674 tablet_replica.cc:439] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0: TxnStatusTablet state changed. Reason: New leader 5b669bbc2aa449b7baf228a3a5c50f6d. Latest consensus state: current_term: 1 leader_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } } }
I20260925 10:55:13.511842 7674 tablet_replica.cc:442] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:55:13.513134 7687 tablet_replica.cc:439] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } } }
I20260925 10:55:13.513374 7687 tablet_replica.cc:442] T f170538f8d8947b8bef344a821a96ef5 P 5b669bbc2aa449b7baf228a3a5c50f6d: This TxnStatusTablet replica's current role is: LEADER
I20260925 10:55:13.515471 7674 tablet_replica.cc:439] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } } }
I20260925 10:55:13.515684 7674 tablet_replica.cc:442] T f170538f8d8947b8bef344a821a96ef5 P a515c5d720cc439aba7d97e1ff90d1e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:55:13.519193 7673 tablet_replica.cc:439] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } } }
I20260925 10:55:13.519464 7673 tablet_replica.cc:442] T f170538f8d8947b8bef344a821a96ef5 P 26571f480c0048dc80063f243ebdcdb6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 10:55:13.532331 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.526223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:34741 (local address 127.31.62.129:45311)
0925 10:55:13.526365 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:13.526369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:13.526948 (+ 579us) server_negotiation.cc:407] Connection header received
0925 10:55:13.527083 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:13.527087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:13.527164 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:13.527258 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:55:13.527733 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.528801 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.530295 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.530936 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.531060 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.531493 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:55:13.531616 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:55:13.531699 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:13.531905 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 10:55:13.532010 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":44}
I20260925 10:55:13.532362 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.526260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:45311 (local address 127.31.62.129:34741)
0925 10:55:13.526854 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:13.526868 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:13.526985 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:13.527302 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:13.527305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:13.527314 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 10:55:13.527601 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.527608 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.528946 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.528949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.530159 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.530171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.531103 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.531107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.531228 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.531315 (+ 87us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 10:55:13.531763 (+ 448us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 10:55:13.531777 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 10:55:13.531867 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 10:55:13.531994 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":2,"thread_start_us":327,"threads_started":1}
I20260925 10:55:13.541026 7457 raft_consensus.cc:1261] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Refusing update from remote peer 5b669bbc2aa449b7baf228a3a5c50f6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:55:13.541116 7591 raft_consensus.cc:1261] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Refusing update from remote peer 5b669bbc2aa449b7baf228a3a5c50f6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:55:13.541608 7677 consensus_queue.cc:1035] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:13.541926 7687 consensus_queue.cc:1035] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:13.551558 7682 mvcc.cc:205] Tried to move back new op lower bound from 7333206890659377152 to 7333206889864404992. Current Snapshot: MvccSnapshot[applied={T|T < 7333206890659377152}]
I20260925 10:55:13.603987 7155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:55:13.604189 7155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:55:13.607416 7155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927: Sending DeleteTablet for 3 replicas of tablet 1fdbbf6df527453d985df21dabf4b7a0
I20260925 10:55:13.608306 7293 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fdbbf6df527453d985df21dabf4b7a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:13 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:53118
I20260925 10:55:13.608371 7432 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fdbbf6df527453d985df21dabf4b7a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:13 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:47354
I20260925 10:55:13.608588 7691 tablet_replica.cc:330] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d: stopping tablet replica
I20260925 10:55:13.608632 7690 tablet_replica.cc:330] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0: stopping tablet replica
I20260925 10:55:13.608783 7691 raft_consensus.cc:2229] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:55:13.608809 7690 raft_consensus.cc:2229] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:13.609001 7690 raft_consensus.cc:2258] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:13.609130 7691 raft_consensus.cc:2258] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:13.609481 7571 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fdbbf6df527453d985df21dabf4b7a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:13 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:60272
I20260925 10:55:13.609743 7692 tablet_replica.cc:330] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6: stopping tablet replica
I20260925 10:55:13.609918 7692 raft_consensus.cc:2229] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:13.610095 7692 raft_consensus.cc:2258] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:13.610395 7691 ts_tablet_manager.cc:1905] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:13.610757 7690 ts_tablet_manager.cc:1905] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:13.611655 7692 ts_tablet_manager.cc:1905] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:13.617964 7691 ts_tablet_manager.cc:1918] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 10:55:13.618175 7691 log.cc:1196] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/1fdbbf6df527453d985df21dabf4b7a0
I20260925 10:55:13.618655 7691 ts_tablet_manager.cc:1939] T 1fdbbf6df527453d985df21dabf4b7a0 P 5b669bbc2aa449b7baf228a3a5c50f6d: Deleting consensus metadata
I20260925 10:55:13.618643 7692 ts_tablet_manager.cc:1918] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 10:55:13.618916 7692 log.cc:1196] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/1fdbbf6df527453d985df21dabf4b7a0
I20260925 10:55:13.619377 7692 ts_tablet_manager.cc:1939] T 1fdbbf6df527453d985df21dabf4b7a0 P 26571f480c0048dc80063f243ebdcdb6: Deleting consensus metadata
I20260925 10:55:13.619840 7139 catalog_manager.cc:4841] TS 5b669bbc2aa449b7baf228a3a5c50f6d (127.31.62.129:45311): tablet 1fdbbf6df527453d985df21dabf4b7a0 (table test-table [id=b3c0f08f36f745bab4f4cbb75f1e7225]) successfully deleted
I20260925 10:55:13.620574 7142 catalog_manager.cc:4841] TS 26571f480c0048dc80063f243ebdcdb6 (127.31.62.131:40519): tablet 1fdbbf6df527453d985df21dabf4b7a0 (table test-table [id=b3c0f08f36f745bab4f4cbb75f1e7225]) successfully deleted
I20260925 10:55:13.626202 7690 ts_tablet_manager.cc:1918] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 10:55:13.626390 7690 log.cc:1196] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/1fdbbf6df527453d985df21dabf4b7a0
I20260925 10:55:13.626816 7690 ts_tablet_manager.cc:1939] T 1fdbbf6df527453d985df21dabf4b7a0 P a515c5d720cc439aba7d97e1ff90d1e0: Deleting consensus metadata
I20260925 10:55:13.628013 7141 catalog_manager.cc:4841] TS a515c5d720cc439aba7d97e1ff90d1e0 (127.31.62.130:44157): tablet 1fdbbf6df527453d985df21dabf4b7a0 (table test-table [id=b3c0f08f36f745bab4f4cbb75f1e7225]) successfully deleted
Sep 25 10:55:13 dist-test-slave-nx7s krb5kdc[7110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 10:55:13 dist-test-slave-nx7s krb5kdc[7110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:13 dist-test-slave-nx7s krb5kdc[7110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
W20260925 10:55:13.670905 7251 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:13.658971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.31.62.190:45847)
0925 10:55:13.659170 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:13.659177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:13.659630 (+ 453us) server_negotiation.cc:407] Connection header received
0925 10:55:13.659846 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:13.659853 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:13.659939 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:13.660042 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:13.661300 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.662400 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.663914 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.664550 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.664690 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.670634 (+ 5944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54352: BlockingRecv error: recv got EOF from 127.0.0.1:54352 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 25 10:55:13 dist-test-slave-nx7s krb5kdc[7110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM
I20260925 10:55:13.689812 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.674063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54364 (local address 127.31.62.190:45847)
0925 10:55:13.674277 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:13.674284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:13.674597 (+ 313us) server_negotiation.cc:407] Connection header received
0925 10:55:13.674789 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:13.674793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:13.674860 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:13.674939 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:13.676761 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.677523 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.679311 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.679777 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.680125 (+ 348us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.684089 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:13.684095 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:13.684098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:13.684137 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:13.686826 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:13.687651 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:13.687658 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:13.687661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:13.687833 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:13.688243 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:13.688248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:13.688251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:13.688608 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:13.688902 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:13.689152 (+ 250us) server_negotiation.cc:299] Negotiation successful
0925 10:55:13.689435 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 10:55:13.693647 7155 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:55:13.694389 7155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:55:13.701515 7432 tablet_service.cc:1467] Processing CreateTablet for tablet e930658aacf342cc8fe9f0d2c9902e9f (DEFAULT_TABLE table=test-table [id=5a17ed342f9b46978d2f70f33e9040f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:13.701488 7293 tablet_service.cc:1467] Processing CreateTablet for tablet e930658aacf342cc8fe9f0d2c9902e9f (DEFAULT_TABLE table=test-table [id=5a17ed342f9b46978d2f70f33e9040f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:13.701927 7432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e930658aacf342cc8fe9f0d2c9902e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:13.701954 7293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e930658aacf342cc8fe9f0d2c9902e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:13.705507 7571 tablet_service.cc:1467] Processing CreateTablet for tablet e930658aacf342cc8fe9f0d2c9902e9f (DEFAULT_TABLE table=test-table [id=5a17ed342f9b46978d2f70f33e9040f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:13.705951 7571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e930658aacf342cc8fe9f0d2c9902e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:13.707635 7668 tablet_bootstrap.cc:492] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0: Bootstrap starting.
I20260925 10:55:13.709276 7668 tablet_bootstrap.cc:654] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:13.711284 7667 tablet_bootstrap.cc:492] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6: Bootstrap starting.
I20260925 10:55:13.711351 7668 tablet_bootstrap.cc:492] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0: No bootstrap required, opened a new log
I20260925 10:55:13.711477 7668 ts_tablet_manager.cc:1397] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 10:55:13.712190 7668 raft_consensus.cc:348] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.712457 7668 raft_consensus.cc:374] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:13.712533 7668 raft_consensus.cc:729] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a515c5d720cc439aba7d97e1ff90d1e0, State: Initialized, Role: FOLLOWER
I20260925 10:55:13.712815 7667 tablet_bootstrap.cc:654] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:13.712773 7668 consensus_queue.cc:260] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.713321 7668 ts_tablet_manager.cc:1428] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 10:55:13.714593 7666 tablet_bootstrap.cc:492] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d: Bootstrap starting.
I20260925 10:55:13.716349 7666 tablet_bootstrap.cc:654] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:13.720598 7667 tablet_bootstrap.cc:492] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6: No bootstrap required, opened a new log
I20260925 10:55:13.720760 7667 ts_tablet_manager.cc:1397] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260925 10:55:13.721467 7667 raft_consensus.cc:348] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.721683 7667 raft_consensus.cc:374] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:13.721764 7667 raft_consensus.cc:729] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26571f480c0048dc80063f243ebdcdb6, State: Initialized, Role: FOLLOWER
I20260925 10:55:13.721992 7667 consensus_queue.cc:260] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.722589 7667 ts_tablet_manager.cc:1428] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 10:55:13.726745 7666 tablet_bootstrap.cc:492] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d: No bootstrap required, opened a new log
I20260925 10:55:13.726907 7666 ts_tablet_manager.cc:1397] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.010s
I20260925 10:55:13.727742 7666 raft_consensus.cc:348] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.728029 7666 raft_consensus.cc:374] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:13.728128 7666 raft_consensus.cc:729] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b669bbc2aa449b7baf228a3a5c50f6d, State: Initialized, Role: FOLLOWER
I20260925 10:55:13.728360 7666 consensus_queue.cc:260] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.728955 7666 ts_tablet_manager.cc:1428] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 10:55:13.943725 7674 raft_consensus.cc:482] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:55:13.943967 7674 raft_consensus.cc:504] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.945802 7674 leader_election.cc:288] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b669bbc2aa449b7baf228a3a5c50f6d (127.31.62.129:45311), 26571f480c0048dc80063f243ebdcdb6 (127.31.62.131:40519)
I20260925 10:55:13.955080 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.946555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:51645 (local address 127.31.62.129:45311)
0925 10:55:13.946734 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:13.946740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:13.947882 (+ 1142us) server_negotiation.cc:407] Connection header received
0925 10:55:13.948178 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:13.948184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:13.948265 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:13.948378 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:13.949493 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.950572 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.952500 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.954013 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.954193 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.954281 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:13.954600 (+ 319us) server_negotiation.cc:299] Negotiation successful
0925 10:55:13.954746 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 10:55:13.955137 7732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.946118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:45311 (local address 127.31.62.130:51645)
0925 10:55:13.947777 (+ 1659us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:13.947796 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:13.947980 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:13.948460 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:13.948464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:13.948485 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:13.949284 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.949302 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.950722 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.950727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.952317 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.952333 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.954198 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.954205 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.954393 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.954435 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 10:55:13.954542 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 10:55:13.954689 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1543,"mutex_wait_us":11,"thread_start_us":609,"threads_started":1}
I20260925 10:55:13.955672 7733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.946105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:40519 (local address 127.31.62.130:39105)
0925 10:55:13.946985 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:13.947098 (+ 113us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:13.947265 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:13.948294 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:13.948299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:13.948334 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:13.948669 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.948681 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.950233 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.950239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.952460 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:13.952478 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.954463 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.954469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:13.954653 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.954691 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 10:55:13.954787 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 10:55:13.954928 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":56,"thread_start_us":1392,"threads_started":1}
I20260925 10:55:13.956054 7734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:13.946301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:39105 (local address 127.31.62.131:40519)
0925 10:55:13.947994 (+ 1693us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:13.948000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:13.948019 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:55:13.948096 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:13.948102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:13.948171 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:13.948264 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:13.948923 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.950046 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.952660 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:13.953967 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:13.954192 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:13.954281 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:13.955227 (+ 946us) server_negotiation.cc:299] Negotiation successful
0925 10:55:13.955380 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1563,"thread_start_us":1460,"threads_started":1}
I20260925 10:55:13.956588 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e930658aacf342cc8fe9f0d2c9902e9f" candidate_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26571f480c0048dc80063f243ebdcdb6" is_pre_election: true
I20260925 10:55:13.956509 7313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e930658aacf342cc8fe9f0d2c9902e9f" candidate_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" is_pre_election: true
I20260925 10:55:13.956909 7590 raft_consensus.cc:2454] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a515c5d720cc439aba7d97e1ff90d1e0 in term 0.
I20260925 10:55:13.956991 7313 raft_consensus.cc:2454] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a515c5d720cc439aba7d97e1ff90d1e0 in term 0.
I20260925 10:55:13.957428 7386 leader_election.cc:302] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26571f480c0048dc80063f243ebdcdb6, a515c5d720cc439aba7d97e1ff90d1e0; no voters:
I20260925 10:55:13.957901 7674 raft_consensus.cc:2781] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:55:13.958045 7674 raft_consensus.cc:482] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:55:13.958148 7674 raft_consensus.cc:3037] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:13.961222 7674 raft_consensus.cc:504] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.962316 7674 leader_election.cc:288] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [CANDIDATE]: Term 1 election: Requested vote from peers 5b669bbc2aa449b7baf228a3a5c50f6d (127.31.62.129:45311), 26571f480c0048dc80063f243ebdcdb6 (127.31.62.131:40519)
I20260925 10:55:13.962491 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e930658aacf342cc8fe9f0d2c9902e9f" candidate_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26571f480c0048dc80063f243ebdcdb6"
I20260925 10:55:13.962699 7590 raft_consensus.cc:3037] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:13.965767 7590 raft_consensus.cc:2454] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a515c5d720cc439aba7d97e1ff90d1e0 in term 1.
I20260925 10:55:13.966482 7313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e930658aacf342cc8fe9f0d2c9902e9f" candidate_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d"
I20260925 10:55:13.966651 7386 leader_election.cc:302] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26571f480c0048dc80063f243ebdcdb6, a515c5d720cc439aba7d97e1ff90d1e0; no voters:
I20260925 10:55:13.966715 7313 raft_consensus.cc:3037] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:13.967114 7674 raft_consensus.cc:2781] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:55:13.970743 7313 raft_consensus.cc:2454] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a515c5d720cc439aba7d97e1ff90d1e0 in term 1.
I20260925 10:55:13.971218 7674 raft_consensus.cc:686] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 LEADER]: Becoming Leader. State: Replica: a515c5d720cc439aba7d97e1ff90d1e0, State: Running, Role: LEADER
I20260925 10:55:13.971665 7674 consensus_queue.cc:237] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:13.996635 7155 catalog_manager.cc:5495] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a515c5d720cc439aba7d97e1ff90d1e0 (127.31.62.130). New cstate: current_term: 1 leader_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } health_report { overall_health: UNKNOWN } } }
I20260925 10:55:14.018748 7738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:14.012767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47368 (local address 127.31.62.130:44157)
0925 10:55:14.013626 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:14.013631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:14.013654 (+ 23us) server_negotiation.cc:407] Connection header received
0925 10:55:14.013771 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:14.013775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:14.013874 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:14.014025 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:55:14.014459 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:14.015217 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:14.016458 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:14.016896 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:14.017010 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:14.017449 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:55:14.018063 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:55:14.018209 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:14.018317 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 10:55:14.018397 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":617,"threads_started":1}
I20260925 10:55:14.022434 7590 raft_consensus.cc:1261] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Refusing update from remote peer a515c5d720cc439aba7d97e1ff90d1e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:55:14.022800 7313 raft_consensus.cc:1261] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 FOLLOWER]: Refusing update from remote peer a515c5d720cc439aba7d97e1ff90d1e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:55:14.023167 7674 consensus_queue.cc:1035] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:14.023504 7735 consensus_queue.cc:1035] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:14.045202 7155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:55:14.045420 7155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:55:14.048300 7155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927: Sending DeleteTablet for 3 replicas of tablet e930658aacf342cc8fe9f0d2c9902e9f
I20260925 10:55:14.049117 7432 tablet_service.cc:1514] Processing DeleteTablet for tablet e930658aacf342cc8fe9f0d2c9902e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:14 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:47354
I20260925 10:55:14.049139 7293 tablet_service.cc:1514] Processing DeleteTablet for tablet e930658aacf342cc8fe9f0d2c9902e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:14 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:53118
I20260925 10:55:14.049436 7690 tablet_replica.cc:330] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0: stopping tablet replica
I20260925 10:55:14.049436 7691 tablet_replica.cc:330] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d: stopping tablet replica
I20260925 10:55:14.049613 7691 raft_consensus.cc:2229] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:14.049613 7690 raft_consensus.cc:2229] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:55:14.049878 7691 raft_consensus.cc:2258] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:14.050160 7690 raft_consensus.cc:2258] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:14.051204 7690 ts_tablet_manager.cc:1905] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:14.051198 7691 ts_tablet_manager.cc:1905] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:14.057832 7571 tablet_service.cc:1514] Processing DeleteTablet for tablet e930658aacf342cc8fe9f0d2c9902e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:14 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:60272
I20260925 10:55:14.058161 7692 tablet_replica.cc:330] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6: stopping tablet replica
I20260925 10:55:14.058333 7692 raft_consensus.cc:2229] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:14.058531 7690 ts_tablet_manager.cc:1918] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:55:14.058593 7692 raft_consensus.cc:2258] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:14.058691 7690 log.cc:1196] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/e930658aacf342cc8fe9f0d2c9902e9f
I20260925 10:55:14.059211 7690 ts_tablet_manager.cc:1939] T e930658aacf342cc8fe9f0d2c9902e9f P a515c5d720cc439aba7d97e1ff90d1e0: Deleting consensus metadata
I20260925 10:55:14.060029 7692 ts_tablet_manager.cc:1905] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:14.060370 7141 catalog_manager.cc:4841] TS a515c5d720cc439aba7d97e1ff90d1e0 (127.31.62.130:44157): tablet e930658aacf342cc8fe9f0d2c9902e9f (table test-table [id=5a17ed342f9b46978d2f70f33e9040f3]) successfully deleted
I20260925 10:55:14.065387 7692 ts_tablet_manager.cc:1918] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:55:14.065593 7692 log.cc:1196] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/e930658aacf342cc8fe9f0d2c9902e9f
I20260925 10:55:14.066042 7692 ts_tablet_manager.cc:1939] T e930658aacf342cc8fe9f0d2c9902e9f P 26571f480c0048dc80063f243ebdcdb6: Deleting consensus metadata
I20260925 10:55:14.066536 7155 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:55:14.067306 7155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:55:14.068127 7691 ts_tablet_manager.cc:1918] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:55:14.068548 7142 catalog_manager.cc:4841] TS 26571f480c0048dc80063f243ebdcdb6 (127.31.62.131:40519): tablet e930658aacf342cc8fe9f0d2c9902e9f (table test-table [id=5a17ed342f9b46978d2f70f33e9040f3]) successfully deleted
I20260925 10:55:14.069522 7691 log.cc:1196] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/e930658aacf342cc8fe9f0d2c9902e9f
I20260925 10:55:14.070169 7691 ts_tablet_manager.cc:1939] T e930658aacf342cc8fe9f0d2c9902e9f P 5b669bbc2aa449b7baf228a3a5c50f6d: Deleting consensus metadata
I20260925 10:55:14.071277 7139 catalog_manager.cc:4841] TS 5b669bbc2aa449b7baf228a3a5c50f6d (127.31.62.129:45311): tablet e930658aacf342cc8fe9f0d2c9902e9f (table test-table [id=5a17ed342f9b46978d2f70f33e9040f3]) successfully deleted
I20260925 10:55:14.074887 7293 tablet_service.cc:1467] Processing CreateTablet for tablet 01dbc71d7a574c30986958523c544303 (DEFAULT_TABLE table=test-table [id=7ddf5f687c3140a881470871816d2e19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:14.075366 7293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01dbc71d7a574c30986958523c544303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:14.075299 7432 tablet_service.cc:1467] Processing CreateTablet for tablet 01dbc71d7a574c30986958523c544303 (DEFAULT_TABLE table=test-table [id=7ddf5f687c3140a881470871816d2e19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:14.075672 7432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01dbc71d7a574c30986958523c544303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:14.077615 7571 tablet_service.cc:1467] Processing CreateTablet for tablet 01dbc71d7a574c30986958523c544303 (DEFAULT_TABLE table=test-table [id=7ddf5f687c3140a881470871816d2e19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:14.078023 7571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01dbc71d7a574c30986958523c544303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:14.085086 7666 tablet_bootstrap.cc:492] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d: Bootstrap starting.
I20260925 10:55:14.086855 7666 tablet_bootstrap.cc:654] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:14.088126 7668 tablet_bootstrap.cc:492] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0: Bootstrap starting.
I20260925 10:55:14.089836 7668 tablet_bootstrap.cc:654] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:14.092211 7667 tablet_bootstrap.cc:492] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6: Bootstrap starting.
I20260925 10:55:14.093955 7667 tablet_bootstrap.cc:654] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:14.094403 7668 tablet_bootstrap.cc:492] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0: No bootstrap required, opened a new log
I20260925 10:55:14.094601 7668 ts_tablet_manager.cc:1397] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260925 10:55:14.095458 7668 raft_consensus.cc:348] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:14.095748 7668 raft_consensus.cc:374] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:14.095844 7668 raft_consensus.cc:729] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a515c5d720cc439aba7d97e1ff90d1e0, State: Initialized, Role: FOLLOWER
I20260925 10:55:14.096071 7666 tablet_bootstrap.cc:492] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d: No bootstrap required, opened a new log
I20260925 10:55:14.096271 7666 ts_tablet_manager.cc:1397] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.002s
I20260925 10:55:14.096135 7668 consensus_queue.cc:260] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:14.097128 7666 raft_consensus.cc:348] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:14.097434 7666 raft_consensus.cc:374] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:14.098157 7666 raft_consensus.cc:729] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b669bbc2aa449b7baf228a3a5c50f6d, State: Initialized, Role: FOLLOWER
I20260925 10:55:14.098503 7666 consensus_queue.cc:260] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:14.099172 7666 ts_tablet_manager.cc:1428] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 10:55:14.099699 7668 ts_tablet_manager.cc:1428] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260925 10:55:14.103772 7667 tablet_bootstrap.cc:492] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6: No bootstrap required, opened a new log
I20260925 10:55:14.104034 7667 ts_tablet_manager.cc:1397] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.003s
I20260925 10:55:14.104823 7667 raft_consensus.cc:348] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:14.105122 7667 raft_consensus.cc:374] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:14.105232 7667 raft_consensus.cc:729] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26571f480c0048dc80063f243ebdcdb6, State: Initialized, Role: FOLLOWER
I20260925 10:55:14.105501 7667 consensus_queue.cc:260] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:14.106055 7667 ts_tablet_manager.cc:1428] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260925 10:55:14.485055 7687 raft_consensus.cc:482] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:55:14.485262 7687 raft_consensus.cc:504] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:14.485926 7687 leader_election.cc:288] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a515c5d720cc439aba7d97e1ff90d1e0 (127.31.62.130:44157), 26571f480c0048dc80063f243ebdcdb6 (127.31.62.131:40519)
I20260925 10:55:14.486509 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01dbc71d7a574c30986958523c544303" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26571f480c0048dc80063f243ebdcdb6" is_pre_election: true
I20260925 10:55:14.486522 7456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01dbc71d7a574c30986958523c544303" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" is_pre_election: true
I20260925 10:55:14.486737 7590 raft_consensus.cc:2454] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 0.
I20260925 10:55:14.486763 7456 raft_consensus.cc:2454] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 0.
I20260925 10:55:14.487241 7246 leader_election.cc:302] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 26571f480c0048dc80063f243ebdcdb6, 5b669bbc2aa449b7baf228a3a5c50f6d, a515c5d720cc439aba7d97e1ff90d1e0; no voters:
I20260925 10:55:14.487550 7687 raft_consensus.cc:2781] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:55:14.487635 7687 raft_consensus.cc:482] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:55:14.487726 7687 raft_consensus.cc:3037] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:14.489912 7687 raft_consensus.cc:504] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:14.490470 7687 leader_election.cc:288] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 election: Requested vote from peers a515c5d720cc439aba7d97e1ff90d1e0 (127.31.62.130:44157), 26571f480c0048dc80063f243ebdcdb6 (127.31.62.131:40519)
I20260925 10:55:14.490935 7456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01dbc71d7a574c30986958523c544303" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a515c5d720cc439aba7d97e1ff90d1e0"
I20260925 10:55:14.491008 7590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01dbc71d7a574c30986958523c544303" candidate_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26571f480c0048dc80063f243ebdcdb6"
I20260925 10:55:14.491163 7456 raft_consensus.cc:3037] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:14.491182 7590 raft_consensus.cc:3037] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:14.494069 7590 raft_consensus.cc:2454] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 1.
I20260925 10:55:14.494164 7456 raft_consensus.cc:2454] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b669bbc2aa449b7baf228a3a5c50f6d in term 1.
I20260925 10:55:14.494599 7245 leader_election.cc:302] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b669bbc2aa449b7baf228a3a5c50f6d, a515c5d720cc439aba7d97e1ff90d1e0; no voters:
I20260925 10:55:14.494874 7687 raft_consensus.cc:2781] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:55:14.495347 7687 raft_consensus.cc:686] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 LEADER]: Becoming Leader. State: Replica: 5b669bbc2aa449b7baf228a3a5c50f6d, State: Running, Role: LEADER
I20260925 10:55:14.495605 7687 consensus_queue.cc:237] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } }
I20260925 10:55:14.498232 7155 catalog_manager.cc:5495] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b669bbc2aa449b7baf228a3a5c50f6d (127.31.62.129). New cstate: current_term: 1 leader_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b669bbc2aa449b7baf228a3a5c50f6d" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 } health_report { overall_health: UNKNOWN } } }
I20260925 10:55:14.567085 7756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:14.561689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53148 (local address 127.31.62.129:45311)
0925 10:55:14.562222 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:14.562227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:14.562329 (+ 102us) server_negotiation.cc:407] Connection header received
0925 10:55:14.562552 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:14.562556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:14.562616 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:14.562732 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:55:14.563147 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:14.563929 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:14.565471 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:14.565871 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:14.565982 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:14.566354 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:55:14.566481 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:55:14.566555 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:14.566692 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 10:55:14.566771 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":270,"threads_started":1}
I20260925 10:55:14.575845 7590 raft_consensus.cc:1261] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Refusing update from remote peer 5b669bbc2aa449b7baf228a3a5c50f6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:55:14.576383 7456 raft_consensus.cc:1261] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Refusing update from remote peer 5b669bbc2aa449b7baf228a3a5c50f6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:55:14.576439 7745 consensus_queue.cc:1035] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "26571f480c0048dc80063f243ebdcdb6" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:14.576893 7687 consensus_queue.cc:1035] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a515c5d720cc439aba7d97e1ff90d1e0" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:14.616832 7153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:55:14.617167 7153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:55:14.619822 7153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b871a10fcdcd48c39c2ccb5b6c819927: Sending DeleteTablet for 3 replicas of tablet 01dbc71d7a574c30986958523c544303
I20260925 10:55:14.620541 7432 tablet_service.cc:1514] Processing DeleteTablet for tablet 01dbc71d7a574c30986958523c544303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:14 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:47354
I20260925 10:55:14.620543 7292 tablet_service.cc:1514] Processing DeleteTablet for tablet 01dbc71d7a574c30986958523c544303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:14 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:53118
I20260925 10:55:14.620656 7571 tablet_service.cc:1514] Processing DeleteTablet for tablet 01dbc71d7a574c30986958523c544303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:14 UTC) from {username='oryx', principal='oryx/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:60272
I20260925 10:55:14.621645 7764 tablet_replica.cc:330] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6: stopping tablet replica
I20260925 10:55:14.621833 7764 raft_consensus.cc:2229] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:14.622023 7764 raft_consensus.cc:2258] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:14.622529 7762 tablet_replica.cc:330] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0: stopping tablet replica
I20260925 10:55:14.622702 7762 raft_consensus.cc:2229] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:14.622710 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 7227
I20260925 10:55:14.622900 7762 raft_consensus.cc:2258] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:14.623008 7764 ts_tablet_manager.cc:1905] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:14.623863 7762 ts_tablet_manager.cc:1905] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:14.629304 7763 tablet_replica.cc:330] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d: stopping tablet replica
I20260925 10:55:14.629513 7763 raft_consensus.cc:2229] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:55:14.629870 7763 raft_consensus.cc:2258] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:14.630034 7762 ts_tablet_manager.cc:1918] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 10:55:14.630165 7762 log.cc:1196] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/01dbc71d7a574c30986958523c544303
I20260925 10:55:14.630559 7762 ts_tablet_manager.cc:1939] T 01dbc71d7a574c30986958523c544303 P a515c5d720cc439aba7d97e1ff90d1e0: Deleting consensus metadata
I20260925 10:55:14.630730 7764 ts_tablet_manager.cc:1918] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 10:55:14.630896 7764 log.cc:1196] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/01dbc71d7a574c30986958523c544303
I20260925 10:55:14.630895 7763 ts_tablet_manager.cc:1905] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:14.631326 7764 ts_tablet_manager.cc:1939] T 01dbc71d7a574c30986958523c544303 P 26571f480c0048dc80063f243ebdcdb6: Deleting consensus metadata
I20260925 10:55:14.632050 7141 catalog_manager.cc:4841] TS a515c5d720cc439aba7d97e1ff90d1e0 (127.31.62.130:44157): tablet 01dbc71d7a574c30986958523c544303 (table test-table [id=7ddf5f687c3140a881470871816d2e19]) successfully deleted
I20260925 10:55:14.632362 7142 catalog_manager.cc:4841] TS 26571f480c0048dc80063f243ebdcdb6 (127.31.62.131:40519): tablet 01dbc71d7a574c30986958523c544303 (table test-table [id=7ddf5f687c3140a881470871816d2e19]) successfully deleted
I20260925 10:55:14.637427 7763 ts_tablet_manager.cc:1918] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 10:55:14.637562 7763 log.cc:1196] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/01dbc71d7a574c30986958523c544303
I20260925 10:55:14.637971 7763 ts_tablet_manager.cc:1939] T 01dbc71d7a574c30986958523c544303 P 5b669bbc2aa449b7baf228a3a5c50f6d: Deleting consensus metadata
Sep 25 10:55:14 dist-test-slave-nx7s krb5kdc[7110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.62.129@KRBTEST.COM
I20260925 10:55:14.638885 7139 catalog_manager.cc:4841] TS 5b669bbc2aa449b7baf228a3a5c50f6d (127.31.62.129:45311): tablet 01dbc71d7a574c30986958523c544303 (table test-table [id=7ddf5f687c3140a881470871816d2e19]) successfully deleted
I20260925 10:55:14.644565 7756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:14.624086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53164 (local address 127.31.62.129:45311)
0925 10:55:14.624253 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:14.624258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:14.628934 (+ 4676us) server_negotiation.cc:407] Connection header received
0925 10:55:14.629405 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:14.629411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:14.629485 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:14.629561 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:14.631210 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:14.632310 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:14.634192 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:14.634647 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:14.634746 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:14.639295 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:14.639310 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:14.639326 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:14.639363 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:14.642209 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:14.642860 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:14.642867 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:14.642870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:14.642995 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:14.643300 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:14.643303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:14.643305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:14.643647 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:14.643859 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:14.644103 (+ 244us) server_negotiation.cc:299] Negotiation successful
0925 10:55:14.644261 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 10:55:14.645299 7358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:14.785459 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 7227
I20260925 10:55:14.806743 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 7367
Sep 25 10:55:14 dist-test-slave-nx7s krb5kdc[7110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.62.130@KRBTEST.COM
I20260925 10:55:14.822810 7767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:14.807586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47382 (local address 127.31.62.130:44157)
0925 10:55:14.808114 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:14.808120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:14.808140 (+ 20us) server_negotiation.cc:407] Connection header received
0925 10:55:14.808204 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:14.808209 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:14.808294 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:14.808407 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:14.809688 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:14.810404 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:14.811677 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:14.812078 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:14.812185 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:14.816426 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:14.816442 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:14.816459 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:14.816495 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:14.819432 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:14.820238 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:14.820247 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:14.820249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:14.820416 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:14.821106 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:14.821111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:14.821113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:14.821576 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:14.821838 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:14.822197 (+ 359us) server_negotiation.cc:299] Negotiation successful
0925 10:55:14.822397 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":246,"threads_started":1}
I20260925 10:55:14.823603 7497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:14.944765 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 7367
I20260925 10:55:14.963128 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 7506
Sep 25 10:55:14 dist-test-slave-nx7s krb5kdc[7110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.62.131@KRBTEST.COM
I20260925 10:55:14.979647 7769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:14.963927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60276 (local address 127.31.62.131:40519)
0925 10:55:14.964425 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:14.964429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:14.964446 (+ 17us) server_negotiation.cc:407] Connection header received
0925 10:55:14.964504 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:14.964508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:14.964569 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:14.964661 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:14.966030 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:14.966798 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:14.968229 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:14.968661 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:14.968765 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:14.973175 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:14.973191 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:14.973208 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:14.973253 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:14.976502 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:14.977417 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:14.977426 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:14.977428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:14.977581 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:14.978005 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:14.978009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:14.978011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:14.978414 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:14.978722 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:14.979051 (+ 329us) server_negotiation.cc:299] Negotiation successful
0925 10:55:14.979278 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":244,"threads_started":1}
I20260925 10:55:14.980435 7635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:15.076537 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 7506
I20260925 10:55:15.090757 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 7123
I20260925 10:55:15.092028 7195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:15.206780 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 7123
2026-09-25T10:55:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6588 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:55:15 dist-test-slave-nx7s krb5kdc[7774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:55:15 dist-test-slave-nx7s krb5kdc[7774](info): set up 2 sockets
Sep 25 10:55:15 dist-test-slave-nx7s krb5kdc[7774](info): commencing operation
krb5kdc: starting...
W20260925 10:55:16.225968 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.977s user 0.003s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:55:16 dist-test-slave-nx7s krb5kdc[7774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:55:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:55:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:55:16.445456 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:42315
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:42315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:55:16.764421 7787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:16.764768 7787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:16.764940 7787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:16.772343 7787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:55:16.772490 7787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:16.772569 7787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:16.772647 7787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:55:16.772715 7787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:55:16.783087 7787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:42315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:42315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:16.784543 7787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:16.786044 7787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:16.798810 7792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:16.798874 7793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:16.798839 7795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:16.799862 7787 server_base.cc:1034] running on GCE node
I20260925 10:55:16.800756 7787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:16.802332 7787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:16.803581 7787 hybrid_clock.cc:648] HybridClock initialized: now 1790333716803530 us; error 43 us; skew 500 ppm
I20260925 10:55:16.803970 7787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-25T10:55:16Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1670 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:55:16 dist-test-slave-nx7s krb5kdc[7801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:55:16 dist-test-slave-nx7s krb5kdc[7801](info): set up 2 sockets
Sep 25 10:55:16 dist-test-slave-nx7s krb5kdc[7801](info): commencing operation
krb5kdc: starting...
W20260925 10:55:17.893996 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.976s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:55:18 dist-test-slave-nx7s krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:55:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:55:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:55:18.111907 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:36505
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:36505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:55:18.438247 7814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:18.438558 7814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:18.438674 7814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:18.445989 7814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:55:18.446111 7814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:18.446156 7814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:18.446205 7814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:55:18.446262 7814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:55:18.456152 7814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:36505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:36505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:18.457515 7814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:18.458876 7814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:18.470952 7822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:18.470932 7820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:18.471041 7819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:18.471633 7814 server_base.cc:1034] running on GCE node
I20260925 10:55:18.472467 7814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:18.474079 7814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:18.475308 7814 hybrid_clock.cc:648] HybridClock initialized: now 1790333718475260 us; error 52 us; skew 500 ppm
I20260925 10:55:18.475694 7814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:18 dist-test-slave-nx7s krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333718, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:18.479008 7814 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:55:18.480602 7814 webserver.cc:460] Webserver started at http://127.31.62.190:35161/ using document root <none> and password file <none>
I20260925 10:55:18.481369 7814 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:18.481491 7814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:18.481791 7814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:18.484920 7814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "829a0d25b0a94d21b2e08fb405704491"
format_stamp: "Formatted at 2026-09-25 10:55:18 on dist-test-slave-nx7s"
server_key: "1caf36445e668ddfd3f5183027f62835"
server_key_iv: "c62b683aae4e6a03fa85a65a123722e4"
server_key_version: "encryptionkey@0"
I20260925 10:55:18.485740 7814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "829a0d25b0a94d21b2e08fb405704491"
format_stamp: "Formatted at 2026-09-25 10:55:18 on dist-test-slave-nx7s"
server_key: "1caf36445e668ddfd3f5183027f62835"
server_key_iv: "c62b683aae4e6a03fa85a65a123722e4"
server_key_version: "encryptionkey@0"
I20260925 10:55:18.491416 7814 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:55:18.495543 7829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:18.497205 7814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260925 10:55:18.497414 7814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "829a0d25b0a94d21b2e08fb405704491"
format_stamp: "Formatted at 2026-09-25 10:55:18 on dist-test-slave-nx7s"
server_key: "1caf36445e668ddfd3f5183027f62835"
server_key_iv: "c62b683aae4e6a03fa85a65a123722e4"
server_key_version: "encryptionkey@0"
I20260925 10:55:18.497620 7814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:18.520030 7814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:18.524381 7814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:18.524653 7814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:18.543278 7814 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:36505
I20260925 10:55:18.543327 7880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:36505 every 8 connection(s)
I20260925 10:55:18.544684 7814 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:55:18.548749 7881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:18.553398 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 7814
I20260925 10:55:18.553627 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:55:18.554378 31994 external_mini_cluster.cc:1428] Setting key 36851c6e744ca7f5f9df321a0ddc021f
I20260925 10:55:18.561102 7881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491: Bootstrap starting.
I20260925 10:55:18.564963 7881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:18.566188 7881 log.cc:824] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:18.569677 7881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491: No bootstrap required, opened a new log
Sep 25 10:55:18 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:55:18.577872 7881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829a0d25b0a94d21b2e08fb405704491" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36505 } }
I20260925 10:55:18.578280 7881 raft_consensus.cc:374] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:18.578379 7881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 829a0d25b0a94d21b2e08fb405704491, State: Initialized, Role: FOLLOWER
I20260925 10:55:18.578926 7884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:18.556591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43496 (local address 127.31.62.190:36505)
0925 10:55:18.557436 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:18.557448 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:18.557497 (+ 49us) server_negotiation.cc:407] Connection header received
0925 10:55:18.559350 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:18.559381 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:18.559870 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:18.560237 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:18.562163 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:18.563761 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:18.565582 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:18.566363 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:18.566521 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:18.571180 (+ 4659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:18.571189 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:18.571209 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:18.571248 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:18.574527 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:18.575254 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:18.575261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:18.575268 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:18.575438 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:18.575827 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:18.575830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:18.575832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:18.576469 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:18.576760 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:18.577326 (+ 566us) server_negotiation.cc:299] Negotiation successful
0925 10:55:18.577744 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":379,"threads_started":1}
I20260925 10:55:18.578948 7881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829a0d25b0a94d21b2e08fb405704491" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36505 } }
I20260925 10:55:18.579254 7881 raft_consensus.cc:388] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:55:18.579394 7881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:55:18.579593 7881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:18.582351 7881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829a0d25b0a94d21b2e08fb405704491" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36505 } }
I20260925 10:55:18.582835 7881 leader_election.cc:302] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 829a0d25b0a94d21b2e08fb405704491; no voters:
I20260925 10:55:18.583364 7881 leader_election.cc:288] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:55:18.583500 7886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:55:18.584141 7886 raft_consensus.cc:686] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 1 LEADER]: Becoming Leader. State: Replica: 829a0d25b0a94d21b2e08fb405704491, State: Running, Role: LEADER
I20260925 10:55:18.584668 7886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829a0d25b0a94d21b2e08fb405704491" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36505 } }
I20260925 10:55:18.585579 7881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:55:18.587422 7887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "829a0d25b0a94d21b2e08fb405704491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829a0d25b0a94d21b2e08fb405704491" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36505 } } }
I20260925 10:55:18.587841 7887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:18.587432 7888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 829a0d25b0a94d21b2e08fb405704491. Latest consensus state: current_term: 1 leader_uuid: "829a0d25b0a94d21b2e08fb405704491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829a0d25b0a94d21b2e08fb405704491" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36505 } } }
I20260925 10:55:18.589293 7891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:55:18.589352 7888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:18.595050 7891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:55:18.605237 7891 catalog_manager.cc:1347] Generated new cluster ID: 4188c478faa041e6bcd3bead6186af79
I20260925 10:55:18.605365 7891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:55:18.628249 7891 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:55:18.629669 7891 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:55:18.640049 7891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491: Generated new TSK 0
I20260925 10:55:18.641232 7891 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:55:18.700958 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:36505
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:55:19.032106 7907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:19.032425 7907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:19.032567 7907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:19.040246 7907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:19.040377 7907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:19.040524 7907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:55:19.051193 7907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:36505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:19.052598 7907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:19.054106 7907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:19.068576 7912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:19.068496 7915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:19.068696 7913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:19.070235 7907 server_base.cc:1034] running on GCE node
I20260925 10:55:19.070989 7907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:19.072007 7907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:19.073235 7907 hybrid_clock.cc:648] HybridClock initialized: now 1790333719073166 us; error 60 us; skew 500 ppm
I20260925 10:55:19.073580 7907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:19 dist-test-slave-nx7s krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333719, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:19.076942 7907 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:55:19.078526 7907 webserver.cc:460] Webserver started at http://127.31.62.129:40117/ using document root <none> and password file <none>
I20260925 10:55:19.079257 7907 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:19.079404 7907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:19.079702 7907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:19.082670 7907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "df52264e1f0846929b3528a6e698afc6"
format_stamp: "Formatted at 2026-09-25 10:55:19 on dist-test-slave-nx7s"
server_key: "34e7bcb3e8426b4a3f3d08f525e1a138"
server_key_iv: "c5cac7dcbdd25bc534df8b330938be04"
server_key_version: "encryptionkey@0"
I20260925 10:55:19.083447 7907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "df52264e1f0846929b3528a6e698afc6"
format_stamp: "Formatted at 2026-09-25 10:55:19 on dist-test-slave-nx7s"
server_key: "34e7bcb3e8426b4a3f3d08f525e1a138"
server_key_iv: "c5cac7dcbdd25bc534df8b330938be04"
server_key_version: "encryptionkey@0"
I20260925 10:55:19.089201 7907 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260925 10:55:19.093079 7922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:19.094609 7907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:55:19.094815 7907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "df52264e1f0846929b3528a6e698afc6"
format_stamp: "Formatted at 2026-09-25 10:55:19 on dist-test-slave-nx7s"
server_key: "34e7bcb3e8426b4a3f3d08f525e1a138"
server_key_iv: "c5cac7dcbdd25bc534df8b330938be04"
server_key_version: "encryptionkey@0"
I20260925 10:55:19.095009 7907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:19.119092 7907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:19.123560 7907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:19.123848 7907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:19.124887 7907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:19.126291 7907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:19.126371 7907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:19.126492 7907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:19.126545 7907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:19.156725 7907 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:44081
I20260925 10:55:19.156800 8034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:44081 every 8 connection(s)
I20260925 10:55:19.158162 7907 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:55:19.162559 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 7907
I20260925 10:55:19.162840 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:55:19.163657 31994 external_mini_cluster.cc:1428] Setting key 1ecd9699c2684160151722df0fcb8b12
Sep 25 10:55:19 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333719, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:55:19.183131 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:19.161157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:59347 (local address 127.31.62.190:36505)
0925 10:55:19.161591 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:19.161596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:19.162946 (+ 1350us) server_negotiation.cc:407] Connection header received
0925 10:55:19.165554 (+ 2608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:19.165560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:19.165686 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:19.165811 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:19.168236 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:19.169145 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:19.171667 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:19.172119 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:19.172262 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:19.176548 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:19.176554 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:19.176557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:19.176597 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:19.179417 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:19.180422 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:19.180426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:19.180464 (+ 38us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:19.180637 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:19.181215 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:19.181219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:19.181221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:19.181527 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:19.181661 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:19.182636 (+ 975us) server_negotiation.cc:299] Negotiation successful
0925 10:55:19.182879 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":225,"threads_started":1}
I20260925 10:55:19.184644 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:19.161704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:59347)
0925 10:55:19.162697 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:19.162773 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:19.165225 (+ 2452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:19.166134 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:19.166147 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:19.167003 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:19.167944 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:19.167983 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:19.169355 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:19.169360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:19.171382 (+ 2022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:19.171405 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:19.172408 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:19.172415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:19.172726 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:19.173551 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:19.173585 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:19.176134 (+ 2549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:19.179612 (+ 3478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:19.179623 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:19.179646 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:19.180263 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:19.180780 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:19.180785 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:19.180788 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:19.181083 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:19.181661 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:19.181669 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:19.182144 (+ 475us) client_negotiation.cc:769] Sending connection context
0925 10:55:19.182475 (+ 331us) client_negotiation.cc:240] Negotiation successful
0925 10:55:19.183093 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":348,"threads_started":1}
I20260925 10:55:19.186398 8035 heartbeater.cc:347] Connected to a master server at 127.31.62.190:36505
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
I20260925 10:55:19.186722 8035 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:19.187448 8035 heartbeater.cc:511] Master 127.31.62.190:36505 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
I20260925 10:55:19.189751 7846 ts_manager.cc:194] Registered new tserver with Master: df52264e1f0846929b3528a6e698afc6 (127.31.62.129:44081)
I20260925 10:55:19.191754 7846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:59347
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:55:19.217809 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:36505
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:55:19.545171 8041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:19.545450 8041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:19.545574 8041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:19.552836 8041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:19.553009 8041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:19.553184 8041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:55:19.563016 8041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:36505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:19.564368 8041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:19.565747 8041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:19.578644 8049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:19.578717 8046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:19.578740 8047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:19.580041 8041 server_base.cc:1034] running on GCE node
I20260925 10:55:19.580732 8041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:19.581750 8041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:19.582959 8041 hybrid_clock.cc:648] HybridClock initialized: now 1790333719582908 us; error 48 us; skew 500 ppm
I20260925 10:55:19.583271 8041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:19 dist-test-slave-nx7s krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333719, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:19.586759 8041 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:55:19.588456 8041 webserver.cc:460] Webserver started at http://127.31.62.130:44065/ using document root <none> and password file <none>
I20260925 10:55:19.589279 8041 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:19.589409 8041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:19.589757 8041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:19.592782 8041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "de1a9c6dbd8744dc89061aa68cae717c"
format_stamp: "Formatted at 2026-09-25 10:55:19 on dist-test-slave-nx7s"
server_key: "326fb9d8c0ea216a9f5ed84fd48bd2ee"
server_key_iv: "342a5e1bf4928e347d75cad50c96e894"
server_key_version: "encryptionkey@0"
I20260925 10:55:19.593633 8041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "de1a9c6dbd8744dc89061aa68cae717c"
format_stamp: "Formatted at 2026-09-25 10:55:19 on dist-test-slave-nx7s"
server_key: "326fb9d8c0ea216a9f5ed84fd48bd2ee"
server_key_iv: "342a5e1bf4928e347d75cad50c96e894"
server_key_version: "encryptionkey@0"
I20260925 10:55:19.599352 8041 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260925 10:55:19.603163 8056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:19.604760 8041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:55:19.604990 8041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "de1a9c6dbd8744dc89061aa68cae717c"
format_stamp: "Formatted at 2026-09-25 10:55:19 on dist-test-slave-nx7s"
server_key: "326fb9d8c0ea216a9f5ed84fd48bd2ee"
server_key_iv: "342a5e1bf4928e347d75cad50c96e894"
server_key_version: "encryptionkey@0"
I20260925 10:55:19.605202 8041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:19.617312 8041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:19.621747 8041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:19.622038 8041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:19.622994 8041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:19.624400 8041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:19.624477 8041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:19.624603 8041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:19.624660 8041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:19.654994 8041 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:37209
I20260925 10:55:19.655031 8168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:37209 every 8 connection(s)
I20260925 10:55:19.656255 8041 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:55:19.658617 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 8041
I20260925 10:55:19.658810 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:55:19.659315 31994 external_mini_cluster.cc:1428] Setting key 184593f2eac00b40b574f265fea1f8c4
Sep 25 10:55:19 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333719, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:55:19.676373 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:19.658775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:41469 (local address 127.31.62.190:36505)
0925 10:55:19.660030 (+ 1255us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:19.660037 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:19.660515 (+ 478us) server_negotiation.cc:407] Connection header received
0925 10:55:19.662368 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:19.662373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:19.662453 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:19.662532 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:19.664027 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:19.664711 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:19.666278 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:19.666707 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:19.666813 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:19.670646 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:19.670651 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:19.670654 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:19.670692 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:19.673227 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:19.674039 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:19.674043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:19.674045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:19.674176 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:19.674602 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:19.674605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:19.674607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:19.674891 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:19.675020 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:19.675785 (+ 765us) server_negotiation.cc:299] Negotiation successful
0925 10:55:19.676024 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1116}
I20260925 10:55:19.677420 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:19.659471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:41469)
0925 10:55:19.660360 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:19.660412 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:19.662104 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:19.662775 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:19.662783 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:19.663305 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:19.663864 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:19.663882 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:19.664908 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:19.664912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:19.666134 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:19.666156 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:19.666918 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:19.666925 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:19.667203 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:19.667868 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:19.667892 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:19.670376 (+ 2484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:19.673397 (+ 3021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:19.673408 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:19.673426 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:19.673902 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:19.674286 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:19.674291 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:19.674295 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:19.674486 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:19.675017 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:19.675026 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:19.675396 (+ 370us) client_negotiation.cc:769] Sending connection context
0925 10:55:19.675656 (+ 260us) client_negotiation.cc:240] Negotiation successful
0925 10:55:19.676256 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":497,"threads_started":1}
I20260925 10:55:19.678989 8169 heartbeater.cc:347] Connected to a master server at 127.31.62.190:36505
I20260925 10:55:19.679308 8169 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
I20260925 10:55:19.680056 8169 heartbeater.cc:511] Master 127.31.62.190:36505 requested a full tablet report, sending...
I20260925 10:55:19.681703 7846 ts_manager.cc:194] Registered new tserver with Master: de1a9c6dbd8744dc89061aa68cae717c (127.31.62.130:37209)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
I20260925 10:55:19.682636 7846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:41469
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:55:19.708423 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:36505
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:55:20.031380 8174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:20.031672 8174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:20.031761 8174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:20.039215 8174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:20.039337 8174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:20.039453 8174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:55:20.049445 8174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:36505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:20.050707 8174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:20.051975 8174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:20.064829 8179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:20.064783 8180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:20.064795 8182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:20.066087 8174 server_base.cc:1034] running on GCE node
I20260925 10:55:20.066754 8174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:20.067771 8174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:20.068960 8174 hybrid_clock.cc:648] HybridClock initialized: now 1790333720068904 us; error 43 us; skew 500 ppm
I20260925 10:55:20.069329 8174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:20 dist-test-slave-nx7s krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333720, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:20.072506 8174 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:55:20.074150 8174 webserver.cc:460] Webserver started at http://127.31.62.131:43563/ using document root <none> and password file <none>
I20260925 10:55:20.074889 8174 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:20.075021 8174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:20.075342 8174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:20.078616 8174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "f8ab3fc639b04086ab87d41456a4089e"
format_stamp: "Formatted at 2026-09-25 10:55:20 on dist-test-slave-nx7s"
server_key: "40431f6da3434d034b165a738e7e3186"
server_key_iv: "79a1af7e8207e1640917524d00f5521a"
server_key_version: "encryptionkey@0"
I20260925 10:55:20.079519 8174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "f8ab3fc639b04086ab87d41456a4089e"
format_stamp: "Formatted at 2026-09-25 10:55:20 on dist-test-slave-nx7s"
server_key: "40431f6da3434d034b165a738e7e3186"
server_key_iv: "79a1af7e8207e1640917524d00f5521a"
server_key_version: "encryptionkey@0"
I20260925 10:55:20.085498 8174 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260925 10:55:20.089389 8189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:20.091001 8174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260925 10:55:20.091169 8174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "f8ab3fc639b04086ab87d41456a4089e"
format_stamp: "Formatted at 2026-09-25 10:55:20 on dist-test-slave-nx7s"
server_key: "40431f6da3434d034b165a738e7e3186"
server_key_iv: "79a1af7e8207e1640917524d00f5521a"
server_key_version: "encryptionkey@0"
I20260925 10:55:20.091374 8174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:20.107872 8174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:20.112375 8174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:20.112663 8174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:20.113684 8174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:20.114982 8174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:20.115058 8174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:20.115170 8174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:20.115221 8174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:20.145735 8174 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:39853
I20260925 10:55:20.145788 8301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:39853 every 8 connection(s)
I20260925 10:55:20.147050 8174 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:55:20.149184 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 8174
I20260925 10:55:20.149351 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:55:20.149816 31994 external_mini_cluster.cc:1428] Setting key 6a69354789696729613c7059a4541bac
Sep 25 10:55:20 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333720, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:55:20.167855 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:20.149889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:50705 (local address 127.31.62.190:36505)
0925 10:55:20.150088 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:20.150094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:20.151538 (+ 1444us) server_negotiation.cc:407] Connection header received
0925 10:55:20.153817 (+ 2279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:20.153822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:20.153910 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:20.153991 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:20.155590 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:20.156479 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:20.158244 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:20.158847 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:20.158987 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:20.162269 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:20.162273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:20.162276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:20.162308 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:20.164915 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:20.165633 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:20.165637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:20.165639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:20.165764 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:20.166170 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:20.166174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:20.166176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:20.166432 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:20.166566 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:20.167292 (+ 726us) server_negotiation.cc:299] Negotiation successful
0925 10:55:20.167516 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 10:55:20.168818 8304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:20.150478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:50705)
0925 10:55:20.151385 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:20.151431 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:20.153548 (+ 2117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:20.154261 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:20.154272 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:20.154821 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:20.155433 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:20.155451 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:20.156653 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:20.156657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:20.158108 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:20.158120 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:20.159042 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:20.159046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:20.159300 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:20.159918 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:20.159941 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:20.162069 (+ 2128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:20.165041 (+ 2972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:20.165051 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:20.165070 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:20.165506 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:20.165862 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:20.165867 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:20.165871 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:20.166057 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:20.166552 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:20.166585 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:20.166931 (+ 346us) client_negotiation.cc:769] Sending connection context
0925 10:55:20.167170 (+ 239us) client_negotiation.cc:240] Negotiation successful
0925 10:55:20.167715 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":330,"threads_started":1}
I20260925 10:55:20.170228 8302 heartbeater.cc:347] Connected to a master server at 127.31.62.190:36505
I20260925 10:55:20.170534 8302 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:20.171295 8302 heartbeater.cc:511] Master 127.31.62.190:36505 requested a full tablet report, sending...
I20260925 10:55:20.172993 7846 ts_manager.cc:194] Registered new tserver with Master: f8ab3fc639b04086ab87d41456a4089e (127.31.62.131:39853)
I20260925 10:55:20.173913 7846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:50705
I20260925 10:55:20.178414 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:55:20.195345 8035 heartbeater.cc:503] Master 127.31.62.190:36505 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.129.keytab.
Entry for principal oryx/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.129.keytab.
WARNING: no policy specified for oryx/127.31.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.130.keytab.
Entry for principal oryx/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.130.keytab.
WARNING: no policy specified for oryx/127.31.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.131.keytab.
Entry for principal oryx/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.131.keytab.
I20260925 10:55:20.313915 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 7907
Sep 25 10:55:20 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:55:20.331455 8311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:20.315043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59050 (local address 127.31.62.129:44081)
0925 10:55:20.315795 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:20.315803 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:20.315986 (+ 183us) server_negotiation.cc:407] Connection header received
0925 10:55:20.316088 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:20.316095 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:20.316338 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:20.316530 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:20.318044 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:20.319110 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:20.320526 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:20.321321 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:20.321429 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:20.325524 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:20.325540 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:20.325555 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:20.325604 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:20.328629 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:20.329415 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:20.329423 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:20.329426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:20.329560 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:20.329959 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:20.329963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:20.329965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:20.330411 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:20.330641 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:20.330924 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 10:55:20.331161 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":379,"threads_started":1}
I20260925 10:55:20.332798 8030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:20.415266 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 7907
I20260925 10:55:20.428483 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 8041
Sep 25 10:55:20 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:55:20.446415 8314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:20.429596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38476 (local address 127.31.62.130:37209)
0925 10:55:20.430107 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:20.430115 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:20.430141 (+ 26us) server_negotiation.cc:407] Connection header received
0925 10:55:20.430204 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:20.430213 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:20.430447 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:20.430597 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:20.431861 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:20.433027 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:20.434400 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:20.434983 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:20.435117 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:20.439683 (+ 4566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:20.439706 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:20.439723 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:20.439767 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:20.443275 (+ 3508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:20.444118 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:20.444127 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:20.444129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:20.444267 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:20.444667 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:20.444671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:20.444673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:20.445114 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:20.445413 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:20.445779 (+ 366us) server_negotiation.cc:299] Negotiation successful
0925 10:55:20.446055 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":241,"threads_started":1}
I20260925 10:55:20.447777 8164 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:20.529668 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 8041
I20260925 10:55:20.542332 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 8174
Sep 25 10:55:20 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:55:20.558992 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:20.543239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54452 (local address 127.31.62.131:39853)
0925 10:55:20.544129 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:20.544135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:20.544172 (+ 37us) server_negotiation.cc:407] Connection header received
0925 10:55:20.544228 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:20.544234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:20.544446 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:20.544589 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:20.545927 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:20.546915 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:20.548166 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:20.548730 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:20.548976 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:20.553395 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:20.553410 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:20.553426 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:20.553468 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:20.556114 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:20.556914 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:20.556923 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:20.556925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:20.557064 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:20.557447 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:20.557452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:20.557455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:20.557811 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:20.558109 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:20.558395 (+ 286us) server_negotiation.cc:299] Negotiation successful
0925 10:55:20.558658 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":585,"threads_started":1}
I20260925 10:55:20.560218 8297 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:20.641508 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 8174
I20260925 10:55:20.654039 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 7814
I20260925 10:55:20.655231 7876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:20.755023 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 7814
I20260925 10:55:20.771642 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:36505
--webserver_interface=127.31.62.190
--webserver_port=35161
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:36505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:55:21.096266 8319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:21.096573 8319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:21.096686 8319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:21.104111 8319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:55:21.104252 8319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:21.104326 8319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:21.104405 8319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:55:21.104455 8319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:55:21.114854 8319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:36505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:36505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=35161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:21.116233 8319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:21.117663 8319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:21.129343 8324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:21.129400 8327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:21.129422 8325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:21.130117 8319 server_base.cc:1034] running on GCE node
I20260925 10:55:21.130934 8319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:21.132402 8319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:21.133622 8319 hybrid_clock.cc:648] HybridClock initialized: now 1790333721133579 us; error 54 us; skew 500 ppm
I20260925 10:55:21.133986 8319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:21 dist-test-slave-nx7s krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333721, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:21.137303 8319 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:55:21.138875 8319 webserver.cc:460] Webserver started at http://127.31.62.190:35161/ using document root <none> and password file <none>
I20260925 10:55:21.139580 8319 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:21.139704 8319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:21.145345 8319 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 10:55:21.148198 8334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:21.149583 8319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:55:21.149731 8319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "829a0d25b0a94d21b2e08fb405704491"
format_stamp: "Formatted at 2026-09-25 10:55:18 on dist-test-slave-nx7s"
server_key: "1caf36445e668ddfd3f5183027f62835"
server_key_iv: "c62b683aae4e6a03fa85a65a123722e4"
server_key_version: "encryptionkey@0"
I20260925 10:55:21.150698 8319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:21.179062 8319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:21.183535 8319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:21.183800 8319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:21.201663 8319 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:36505
I20260925 10:55:21.201696 8385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:36505 every 8 connection(s)
I20260925 10:55:21.203117 8319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:55:21.210073 8386 sys_catalog.cc:263] Verifying existing consensus state
I20260925 10:55:21.213485 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 8319
I20260925 10:55:21.214123 8386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491: Bootstrap starting.
I20260925 10:55:21.214429 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:44081
--local_ip_for_outbound_sockets=127.31.62.129
--tserver_master_addrs=127.31.62.190:36505
--webserver_port=40117
--webserver_interface=127.31.62.129
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.62.129
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:55:21.234998 8386 log.cc:824] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:21.244415 8386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 10:55:21.245007 8386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491: Bootstrap complete.
I20260925 10:55:21.253978 8386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829a0d25b0a94d21b2e08fb405704491" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36505 } }
I20260925 10:55:21.254878 8386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 829a0d25b0a94d21b2e08fb405704491, State: Initialized, Role: FOLLOWER
I20260925 10:55:21.255551 8386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829a0d25b0a94d21b2e08fb405704491" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36505 } }
I20260925 10:55:21.255781 8386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:55:21.255889 8386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:55:21.256038 8386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 1 FOLLOWER]: Advancing to term 2
I20260925 10:55:21.259806 8386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829a0d25b0a94d21b2e08fb405704491" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36505 } }
I20260925 10:55:21.260285 8386 leader_election.cc:302] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 829a0d25b0a94d21b2e08fb405704491; no voters:
I20260925 10:55:21.260910 8386 leader_election.cc:288] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 10:55:21.261073 8390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 10:55:21.262163 8390 raft_consensus.cc:686] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [term 2 LEADER]: Becoming Leader. State: Replica: 829a0d25b0a94d21b2e08fb405704491, State: Running, Role: LEADER
I20260925 10:55:21.262794 8390 consensus_queue.cc:237] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829a0d25b0a94d21b2e08fb405704491" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36505 } }
I20260925 10:55:21.263252 8386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:55:21.265618 8392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 829a0d25b0a94d21b2e08fb405704491. Latest consensus state: current_term: 2 leader_uuid: "829a0d25b0a94d21b2e08fb405704491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829a0d25b0a94d21b2e08fb405704491" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36505 } } }
I20260925 10:55:21.266101 8391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "829a0d25b0a94d21b2e08fb405704491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829a0d25b0a94d21b2e08fb405704491" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36505 } } }
I20260925 10:55:21.266462 8391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:21.266126 8392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:21.279381 8397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:55:21.285796 8397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:55:21.287312 8397 catalog_manager.cc:1259] Loaded cluster ID: 4188c478faa041e6bcd3bead6186af79
I20260925 10:55:21.287492 8397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:55:21.291554 8397 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:55:21.293211 8397 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 829a0d25b0a94d21b2e08fb405704491: Loaded TSK: 0
I20260925 10:55:21.295054 8397 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 10:55:21.563058 8388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:21.563372 8388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:21.563512 8388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:21.571060 8388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:21.571204 8388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:21.571357 8388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:55:21.581684 8388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.129.keytab
--principal=oryx/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:44081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=40117
--tserver_master_addrs=127.31.62.190:36505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:21.583009 8388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:21.584318 8388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:21.598883 8413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:21.599159 8415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:21.598910 8412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:21.600044 8388 server_base.cc:1034] running on GCE node
I20260925 10:55:21.600881 8388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:21.602113 8388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:21.603348 8388 hybrid_clock.cc:648] HybridClock initialized: now 1790333721603296 us; error 48 us; skew 500 ppm
I20260925 10:55:21.603701 8388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:21 dist-test-slave-nx7s krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333721, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:21.607700 8388 init.cc:377] Logged in from keytab as oryx/127.31.62.129@KRBTEST.COM (short username oryx)
I20260925 10:55:21.609543 8388 webserver.cc:460] Webserver started at http://127.31.62.129:40117/ using document root <none> and password file <none>
I20260925 10:55:21.610472 8388 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:21.610630 8388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:21.617587 8388 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 10:55:21.620884 8422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:21.622613 8388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:55:21.622795 8388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "df52264e1f0846929b3528a6e698afc6"
format_stamp: "Formatted at 2026-09-25 10:55:19 on dist-test-slave-nx7s"
server_key: "34e7bcb3e8426b4a3f3d08f525e1a138"
server_key_iv: "c5cac7dcbdd25bc534df8b330938be04"
server_key_version: "encryptionkey@0"
I20260925 10:55:21.624071 8388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:21.643914 8388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:21.647992 8388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:21.648216 8388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:21.649150 8388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:21.650461 8388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:21.650552 8388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:21.650628 8388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:21.650682 8388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:21.680552 8388 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:44081
I20260925 10:55:21.680593 8534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:44081 every 8 connection(s)
I20260925 10:55:21.682045 8388 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:55:21.689024 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 8388
I20260925 10:55:21.689616 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:37209
--local_ip_for_outbound_sockets=127.31.62.130
--tserver_master_addrs=127.31.62.190:36505
--webserver_port=44065
--webserver_interface=127.31.62.130
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.62.130
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 10:55:21 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:21 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
W20260925 10:55:22.015760 8539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:22.016054 8539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:22.016181 8539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:22.023698 8539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:22.023818 8539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:22.023957 8539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:55:22.034019 8539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.130.keytab
--principal=oryx/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:37209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=44065
--tserver_master_addrs=127.31.62.190:36505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:22.035379 8539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:22.036654 8539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:22.052479 8546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:22.052779 8545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:22.053632 8548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:22.053754 8539 server_base.cc:1034] running on GCE node
I20260925 10:55:22.054813 8539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:22.056358 8539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:22.057619 8539 hybrid_clock.cc:648] HybridClock initialized: now 1790333722057533 us; error 66 us; skew 500 ppm
I20260925 10:55:22.058106 8539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:22 dist-test-slave-nx7s krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333722, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:22.065320 8539 init.cc:377] Logged in from keytab as oryx/127.31.62.130@KRBTEST.COM (short username oryx)
I20260925 10:55:22.067445 8539 webserver.cc:460] Webserver started at http://127.31.62.130:44065/ using document root <none> and password file <none>
I20260925 10:55:22.068534 8539 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:22.068732 8539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:22.077719 8539 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.000s
I20260925 10:55:22.082554 8555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:22.084892 8539 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:55:22.085172 8539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "de1a9c6dbd8744dc89061aa68cae717c"
format_stamp: "Formatted at 2026-09-25 10:55:19 on dist-test-slave-nx7s"
server_key: "326fb9d8c0ea216a9f5ed84fd48bd2ee"
server_key_iv: "342a5e1bf4928e347d75cad50c96e894"
server_key_version: "encryptionkey@0"
I20260925 10:55:22.086885 8539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:22.118320 8539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:22.125052 8539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:22.125460 8539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:22.126919 8539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:22.129150 8539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:22.129305 8539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:22.129451 8539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:22.129528 8539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:22.165335 8539 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:37209
I20260925 10:55:22.165347 8667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:37209 every 8 connection(s)
I20260925 10:55:22.167490 8539 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:55:22.173431 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 8539
I20260925 10:55:22.174131 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:39853
--local_ip_for_outbound_sockets=127.31.62.131
--tserver_master_addrs=127.31.62.190:36505
--webserver_port=43563
--webserver_interface=127.31.62.131
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.62.131
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 10:55:22 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:22 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
W20260925 10:55:22.531474 8672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:22.531777 8672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:22.531929 8672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:22.539219 8672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:22.539362 8672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:22.539562 8672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:55:22.549791 8672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/krb5kdc/oryx_127.31.62.131.keytab
--principal=oryx/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:39853
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=43563
--tserver_master_addrs=127.31.62.190:36505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:22.551234 8672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:22.552565 8672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:22.568697 8678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:22.576892 8679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:22.579174 8672 server_base.cc:1034] running on GCE node
W20260925 10:55:22.579384 8681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:22.580163 8672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:22.581519 8672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:22.582801 8672 hybrid_clock.cc:648] HybridClock initialized: now 1790333722582739 us; error 51 us; skew 500 ppm
I20260925 10:55:22.583140 8672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:22 dist-test-slave-nx7s krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333722, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:22.586303 8672 init.cc:377] Logged in from keytab as oryx/127.31.62.131@KRBTEST.COM (short username oryx)
I20260925 10:55:22.590189 8672 webserver.cc:460] Webserver started at http://127.31.62.131:43563/ using document root <none> and password file <none>
I20260925 10:55:22.590924 8672 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:22.591065 8672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:22.597189 8672 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 10:55:22.599901 8688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:22.601347 8672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:55:22.601508 8672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "f8ab3fc639b04086ab87d41456a4089e"
format_stamp: "Formatted at 2026-09-25 10:55:20 on dist-test-slave-nx7s"
server_key: "40431f6da3434d034b165a738e7e3186"
server_key_iv: "79a1af7e8207e1640917524d00f5521a"
server_key_version: "encryptionkey@0"
I20260925 10:55:22.602473 8672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:22.617821 8672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:22.622068 8672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:22.622347 8672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:22.623893 8672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:22.625448 8672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:22.625545 8672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:22.625646 8672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:22.625715 8672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:22.667510 8672 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:39853
I20260925 10:55:22.667555 8800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:39853 every 8 connection(s)
I20260925 10:55:22.669304 8672 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:55:22.674068 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 8672
Sep 25 10:55:22 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:22 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:22.716475 8804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:22.675923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43504 (local address 127.31.62.190:36505)
0925 10:55:22.693993 (+ 18070us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:22.694000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:22.694023 (+ 23us) server_negotiation.cc:407] Connection header received
0925 10:55:22.694374 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:22.694380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:22.694464 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:22.694561 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:22.697002 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.698325 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.701515 (+ 3190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.702130 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.702343 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:22.705675 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:22.705690 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:22.705712 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:22.705758 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:22.710004 (+ 4246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:22.711133 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:22.711142 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:22.711151 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:22.711349 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:22.712215 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:22.712220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:22.712223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:22.713281 (+ 1058us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:22.713574 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:22.715008 (+ 1434us) server_negotiation.cc:299] Negotiation successful
0925 10:55:22.715467 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":17945,"thread_start_us":16800,"threads_started":1}
W20260925 10:55:22.933418 8538 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.236s user 0.007s sys 0.005s
W20260925 10:55:22.935130 8537 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:21.685326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:40393 (local address 127.31.62.190:36505)
0925 10:55:21.686002 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:21.686015 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:21.686491 (+ 476us) server_negotiation.cc:407] Connection header received
0925 10:55:21.688698 (+ 2207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:21.688715 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:21.689273 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:21.689609 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:21.691660 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:21.693153 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:21.695458 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:21.696409 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:21.696616 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:22.934744 (+1238128us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:40393: BlockingRecv error: recv got EOF from 127.31.62.129:40393 (error 108)
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":261,"threads_started":1}
I20260925 10:55:22.935731 8538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:21.685450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:40393)
0925 10:55:21.686260 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:21.686307 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:21.688083 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:21.689871 (+ 1788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:21.689883 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:21.690597 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:21.691402 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:21.691427 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:21.693349 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:21.693355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:21.695252 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:21.695271 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:21.696659 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:21.696665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:21.696922 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:21.697603 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:21.697625 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:22.934426 (+1236801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":339,"threads_started":1}
W20260925 10:55:22.936973 8535 heartbeater.cc:644] Failed to heartbeat to 127.31.62.190:36505 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.62.190:36505: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Sep 25 10:55:22 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:22 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:22.951330 8538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:22.937765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:44003)
0925 10:55:22.938297 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:22.938319 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:22.938475 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:22.938863 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:22.938868 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:22.939388 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:22.939628 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:22.939640 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.941540 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.941548 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:22.943274 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:22.943296 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.944770 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.944778 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:22.945015 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:22.945792 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:22.945821 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:22.950948 (+ 5127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280}
W20260925 10:55:22.951622 8537 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:22.937829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:44003 (local address 127.31.62.190:36505)
0925 10:55:22.938015 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:22.938022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:22.938421 (+ 399us) server_negotiation.cc:407] Connection header received
0925 10:55:22.938608 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:22.938614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:22.938707 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:22.938822 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:22.939793 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.941341 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.943508 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.944286 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.944396 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:22.951314 (+ 6918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:44003: BlockingRecv error: recv got EOF from 127.31.62.129:44003 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 25 10:55:22 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:22 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
W20260925 10:55:22.966105 8537 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:22.953093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:39279 (local address 127.31.62.190:36505)
0925 10:55:22.953300 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:22.953308 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:22.953364 (+ 56us) server_negotiation.cc:407] Connection header received
0925 10:55:22.953440 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:22.953446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:22.953557 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:22.953681 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:22.954761 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.956214 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.958413 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.959172 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.959277 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:22.965753 (+ 6476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:39279: BlockingRecv error: recv got EOF from 127.31.62.129:39279 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 10:55:22.966801 8538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:22.952610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:39279)
0925 10:55:22.952807 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:22.952832 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:22.953268 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:22.953724 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:22.953730 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:22.954255 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:22.954592 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:22.954609 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.956436 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.956444 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:22.958218 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:22.958236 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.959577 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.959584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:22.959813 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:22.960599 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:22.960625 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:22.966488 (+ 5863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260925 10:55:22.967269 8535 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 10:55:23.369673 8671 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.184s user 0.008s sys 0.008s
W20260925 10:55:23.371356 8670 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:22.170554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:36057 (local address 127.31.62.190:36505)
0925 10:55:22.170973 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:22.170979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:22.172492 (+ 1513us) server_negotiation.cc:407] Connection header received
0925 10:55:22.175041 (+ 2549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:22.175049 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:22.175160 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:22.175301 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:22.177508 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.178840 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.181217 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.181621 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.181751 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:23.371043 (+1189292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:36057: BlockingRecv error: recv got EOF from 127.31.62.130:36057 (error 108)
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":222,"threads_started":1}
I20260925 10:55:23.372418 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:22.171410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:36057)
0925 10:55:22.172317 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:22.172363 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:22.174619 (+ 2256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:22.175621 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:22.175633 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:22.176421 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:22.177286 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:22.177313 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.179056 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.179061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:22.181041 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:22.181061 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.181881 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.181888 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:22.182134 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:22.185127 (+ 2993us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:22.185166 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:23.370888 (+1185722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":345,"threads_started":1}
W20260925 10:55:23.373433 8668 heartbeater.cc:644] Failed to heartbeat to 127.31.62.190:36505 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.62.190:36505: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Sep 25 10:55:23 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:23 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:23.384449 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:23.374116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:34329)
0925 10:55:23.374313 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:23.374329 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:23.374429 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:23.374884 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:23.374887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:23.375243 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:23.375411 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:23.375419 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.377162 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.377167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:23.378205 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:23.378220 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.379359 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.379365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:23.379494 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:23.380237 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:23.380259 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:23.384223 (+ 3964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260925 10:55:23.384820 8670 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:23.374217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:34329 (local address 127.31.62.190:36505)
0925 10:55:23.374421 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:23.374428 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:23.374455 (+ 27us) server_negotiation.cc:407] Connection header received
0925 10:55:23.374561 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:23.374566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:23.374667 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:23.374781 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:23.375559 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.376948 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.378359 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.379162 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.379310 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:23.384514 (+ 5204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:34329: BlockingRecv error: recv got EOF from 127.31.62.130:34329 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 25 10:55:23 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:23 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
W20260925 10:55:23.397075 8670 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:23.385463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:42325 (local address 127.31.62.190:36505)
0925 10:55:23.385596 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:23.385600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:23.385614 (+ 14us) server_negotiation.cc:407] Connection header received
0925 10:55:23.385665 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:23.385668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:23.385741 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:23.385818 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:23.386807 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.387569 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.389786 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.390217 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.390329 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:23.396749 (+ 6420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:42325: BlockingRecv error: recv got EOF from 127.31.62.130:42325 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 10:55:23.397683 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:23.385164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:42325)
0925 10:55:23.385372 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:23.385396 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:23.385530 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:23.385883 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:23.385889 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:23.386350 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:23.386620 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:23.386635 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.387742 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.387748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:23.389556 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:23.389580 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.390383 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.390389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:23.390595 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:23.391386 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:23.391412 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:23.397484 (+ 6072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260925 10:55:23.397969 8668 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 10:55:23.720659 8803 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.010s user 0.000s sys 0.013s
W20260925 10:55:23.721877 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:22.674883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:58157 (local address 127.31.62.190:36505)
0925 10:55:22.692987 (+ 18104us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:22.692994 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:22.699659 (+ 6665us) server_negotiation.cc:407] Connection header received
0925 10:55:22.701872 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:22.701878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:22.701962 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:22.702050 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:22.704893 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.706076 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.708525 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.709145 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.709236 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:23.721636 (+1012400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:58157: BlockingRecv error: recv got EOF from 127.31.62.131:58157 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":17944,"thread_start_us":18863,"threads_started":1}
I20260925 10:55:23.722404 8803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:22.673028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:58157)
0925 10:55:22.699066 (+ 26038us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:22.699123 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:22.701554 (+ 2431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:22.702949 (+ 1395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:22.702962 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:22.703759 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:22.704642 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:22.704669 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.706289 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.706295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:22.708365 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:22.708382 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:22.709595 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:22.709601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:22.709892 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:22.710531 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:22.710561 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:23.721434 (+1010873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25702,"thread_start_us":25453,"threads_started":1}
W20260925 10:55:23.723302 8801 heartbeater.cc:644] Failed to heartbeat to 127.31.62.190:36505 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.62.190:36505: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Sep 25 10:55:23 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:23 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:23.732965 8803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:23.723983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:45695)
0925 10:55:23.724167 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:23.724187 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:23.724307 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:23.724682 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:23.724688 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:23.725175 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:23.725368 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:23.725383 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.726557 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.726561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:23.727585 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:23.727598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.728311 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.728316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:23.728432 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:23.729068 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:23.729086 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:23.732715 (+ 3629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260925 10:55:23.733266 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:23.724156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:45695 (local address 127.31.62.190:36505)
0925 10:55:23.724303 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:23.724307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:23.724325 (+ 18us) server_negotiation.cc:407] Connection header received
0925 10:55:23.724416 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:23.724421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:23.724491 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:23.724626 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:23.725528 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.726419 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.727761 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.728163 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.728253 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:23.733042 (+ 4789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:45695: BlockingRecv error: recv got EOF from 127.31.62.131:45695 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 25 10:55:23 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:23 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:23.742522 8803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:23.733835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:48875)
0925 10:55:23.733980 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:23.734000 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:23.734115 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:23.734499 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:23.734502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:23.734799 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:23.734959 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:23.734966 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.736033 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.736037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:23.737411 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:23.737427 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.738258 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.738264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:23.738448 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:23.738993 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:23.739010 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:23.742245 (+ 3235us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 10:55:23.742689 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:23.733919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:48875 (local address 127.31.62.190:36505)
0925 10:55:23.734246 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:23.734250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:23.734266 (+ 16us) server_negotiation.cc:407] Connection header received
0925 10:55:23.734313 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:23.734316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:23.734385 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:23.734463 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:23.735124 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.735860 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.737562 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.738078 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.738169 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:23.742477 (+ 4308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:48875: BlockingRecv error: recv got EOF from 127.31.62.131:48875 (error 108)
Metrics: {"server-negotiator.queue_time_us":226}
W20260925 10:55:23.742893 8801 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 25 10:55:23 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:23 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:23.977149 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:23.967841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:42385)
0925 10:55:23.968374 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:23.968390 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:23.968505 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:23.968812 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:23.968815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:23.969263 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:23.969414 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:23.969423 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.970477 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.970483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:23.971742 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:23.971759 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.972561 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.972566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:23.972696 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:23.973304 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:23.973323 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:23.976895 (+ 3572us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":288,"threads_started":1}
W20260925 10:55:23.977327 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:23.967937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:42385 (local address 127.31.62.190:36505)
0925 10:55:23.968099 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:23.968103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:23.968464 (+ 361us) server_negotiation.cc:407] Connection header received
0925 10:55:23.968611 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:23.968614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:23.968694 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:23.968783 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:23.969557 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.970272 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.971890 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:23.972389 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:23.972482 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:23.977128 (+ 4646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:42385: BlockingRecv error: recv got EOF from 127.31.62.129:42385 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 25 10:55:24 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:24 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:24.408088 8809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:24.398507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:40285)
0925 10:55:24.399020 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:24.399041 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:24.399158 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:24.399548 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:24.399552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:24.399961 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:24.400129 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:24.400139 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:24.401222 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:24.401226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:24.402235 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:24.402247 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:24.402975 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:24.402980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:24.403135 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:24.403743 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:24.403761 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:24.407759 (+ 3998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":277,"threads_started":1}
W20260925 10:55:24.408190 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:24.398632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:40285 (local address 127.31.62.190:36505)
0925 10:55:24.398822 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:24.398828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:24.399121 (+ 293us) server_negotiation.cc:407] Connection header received
0925 10:55:24.399306 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:24.399311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:24.399377 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:24.399461 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:24.400289 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:24.401083 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:24.402353 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:24.402789 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:24.402886 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:24.407971 (+ 5085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:40285: BlockingRecv error: recv got EOF from 127.31.62.130:40285 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 25 10:55:24 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:24 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:24.752954 8810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:24.743576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:51069)
0925 10:55:24.744213 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:24.744229 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:24.744336 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:24.744643 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:24.744646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:24.745223 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:24.745369 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:24.745377 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:24.746428 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:24.746432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:24.747400 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:24.747412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:24.748230 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:24.748235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:24.748368 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:24.749091 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:24.749113 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:24.752679 (+ 3566us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":352,"threads_started":1}
W20260925 10:55:24.753223 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:24.743621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:51069 (local address 127.31.62.190:36505)
0925 10:55:24.743788 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:24.743794 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:24.744338 (+ 544us) server_negotiation.cc:407] Connection header received
0925 10:55:24.744416 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:24.744420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:24.744512 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:24.744595 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:24.745506 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:24.746299 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:24.747578 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:24.747994 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:24.748117 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:24.752998 (+ 4881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:51069: BlockingRecv error: recv got EOF from 127.31.62.131:51069 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 25 10:55:24 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:24 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:24.987354 8811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:24.978112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:52315)
0925 10:55:24.978783 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:24.978799 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:24.978922 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:24.979280 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:24.979285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:24.979733 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:24.979954 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:24.979964 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:24.981144 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:24.981148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:24.982053 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:24.982064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:24.982831 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:24.982837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:24.982986 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:24.983663 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:24.983682 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:24.987103 (+ 3421us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":275,"threads_started":1}
W20260925 10:55:24.987633 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:24.978261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:52315 (local address 127.31.62.190:36505)
0925 10:55:24.978482 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:24.978488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:24.978896 (+ 408us) server_negotiation.cc:407] Connection header received
0925 10:55:24.979033 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:24.979038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:24.979103 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:24.979190 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:24.980104 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:24.981023 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:24.982208 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:24.982656 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:24.982757 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:24.987366 (+ 4609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:52315: BlockingRecv error: recv got EOF from 127.31.62.129:52315 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 25 10:55:25 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:25 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:25.417912 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:25.409033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:52055)
0925 10:55:25.409507 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:25.409528 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:25.409703 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:25.409989 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:25.409994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:25.410403 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:25.410583 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:25.410592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:25.411633 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:25.411638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:25.412566 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:25.412578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:25.413315 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:25.413320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:25.413480 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:25.414042 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:25.414062 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:25.417665 (+ 3603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":252,"threads_started":1}
W20260925 10:55:25.418216 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:25.409123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:52055 (local address 127.31.62.190:36505)
0925 10:55:25.409323 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:25.409330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:25.409631 (+ 301us) server_negotiation.cc:407] Connection header received
0925 10:55:25.409813 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:25.409817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:25.409875 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:25.409951 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:25.410716 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:25.411496 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:25.412696 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:25.413166 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:25.413275 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:25.417917 (+ 4642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:52055: BlockingRecv error: recv got EOF from 127.31.62.130:52055 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 25 10:55:25 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:25 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:25.763382 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:25.753991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:36165)
0925 10:55:25.754541 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:25.754565 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:25.754680 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:25.754997 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:25.755000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:25.755363 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:25.755546 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:25.755584 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:25.756781 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:25.756786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:25.758023 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:25.758035 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:25.758709 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:25.758714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:25.758848 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:25.759426 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:25.759446 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:25.763000 (+ 3554us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":332,"threads_started":1}
W20260925 10:55:25.763571 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:25.754271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:36165 (local address 127.31.62.190:36505)
0925 10:55:25.754439 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:25.754443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:25.754633 (+ 190us) server_negotiation.cc:407] Connection header received
0925 10:55:25.754781 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:25.754786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:25.754870 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:25.754978 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:25.755702 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:25.756641 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:25.758145 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:25.758505 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:25.758602 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:25.763333 (+ 4731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:36165: BlockingRecv error: recv got EOF from 127.31.62.131:36165 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 10:55:25 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:25 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:25.997327 8814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:25.988328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:45397)
0925 10:55:25.988875 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:25.988894 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:25.989011 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:25.989320 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:25.989323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:25.989638 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:25.989827 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:25.989836 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:25.990893 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:25.990897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:25.991925 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:25.991940 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:25.992935 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:25.992940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:25.993100 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:25.993791 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:25.993809 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:25.997087 (+ 3278us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":270,"threads_started":1}
W20260925 10:55:25.997531 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:25.988438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:45397 (local address 127.31.62.190:36505)
0925 10:55:25.988598 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:25.988603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:25.988962 (+ 359us) server_negotiation.cc:407] Connection header received
0925 10:55:25.989111 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:25.989114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:25.989176 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:25.989277 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:25.989951 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:25.990691 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:25.992141 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:25.992735 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:25.992888 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:25.997329 (+ 4441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:45397: BlockingRecv error: recv got EOF from 127.31.62.129:45397 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 25 10:55:26 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:26 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:26.427891 8815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:26.418844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:54773)
0925 10:55:26.419434 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:26.419453 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:26.419583 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:26.419905 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:26.419909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:26.420256 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:26.420430 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:26.420439 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:26.421547 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:26.421550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:26.422479 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:26.422491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:26.423156 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:26.423162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:26.423318 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:26.423927 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:26.423945 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:26.427645 (+ 3700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":359,"threads_started":1}
W20260925 10:55:26.428115 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:26.418920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:54773 (local address 127.31.62.190:36505)
0925 10:55:26.419092 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:26.419098 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:26.419541 (+ 443us) server_negotiation.cc:407] Connection header received
0925 10:55:26.419699 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:26.419703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:26.419766 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:26.419844 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:26.420607 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:26.421400 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:26.422626 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:26.423017 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:26.423126 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:26.427875 (+ 4749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:54773: BlockingRecv error: recv got EOF from 127.31.62.130:54773 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 25 10:55:26 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:26 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:26.773129 8816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:26.764460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:39097)
0925 10:55:26.765015 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:26.765032 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:26.765173 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:26.765523 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:26.765528 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:26.765880 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:26.766046 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:26.766055 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:26.767040 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:26.767045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:26.767921 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:26.767933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:26.768616 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:26.768622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:26.768774 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:26.769443 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:26.769460 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:26.772889 (+ 3429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":265,"threads_started":1}
W20260925 10:55:26.773463 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:26.764593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:39097 (local address 127.31.62.190:36505)
0925 10:55:26.764824 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:26.764829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:26.765123 (+ 294us) server_negotiation.cc:407] Connection header received
0925 10:55:26.765303 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:26.765307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:26.765384 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:26.765465 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:26.766191 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:26.766904 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:26.768088 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:26.768488 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:26.768590 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:26.773189 (+ 4599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:39097: BlockingRecv error: recv got EOF from 127.31.62.131:39097 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Sep 25 10:55:27 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:27 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:27.008010 8817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:26.998346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:35897)
0925 10:55:26.998813 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:26.998829 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:26.998942 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:26.999288 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:26.999291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:26.999638 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:26.999798 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:26.999805 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:27.000887 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:27.000891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:27.001903 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:27.001915 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:27.002709 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:27.002714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:27.002859 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:27.003543 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:27.003566 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:27.007658 (+ 4092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":258,"threads_started":1}
W20260925 10:55:27.008149 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:26.998421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:35897 (local address 127.31.62.190:36505)
0925 10:55:26.998651 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:26.998658 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:26.998904 (+ 246us) server_negotiation.cc:407] Connection header received
0925 10:55:26.999051 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:26.999055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:26.999116 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:26.999226 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:26.999967 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:27.000730 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:27.002113 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:27.002526 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:27.002654 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:27.007873 (+ 5219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:35897: BlockingRecv error: recv got EOF from 127.31.62.129:35897 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Sep 25 10:55:27 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:27 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:27.444612 8818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:27.429081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:54121)
0925 10:55:27.429634 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:27.429670 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:27.429797 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:27.430201 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:27.430205 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:27.430635 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:27.430811 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:27.430820 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:27.431813 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:27.431817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:27.433129 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:27.433143 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:27.433982 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:27.433988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:27.434172 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:27.440827 (+ 6655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:27.440895 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:27.444353 (+ 3458us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":254,"threads_started":1}
W20260925 10:55:27.444945 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:27.429359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:54121 (local address 127.31.62.190:36505)
0925 10:55:27.429570 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:27.429574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:27.429760 (+ 186us) server_negotiation.cc:407] Connection header received
0925 10:55:27.429897 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:27.429901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:27.429964 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:27.430048 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:27.430957 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:27.431679 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:27.433285 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:27.433838 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:27.433954 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:27.444637 (+ 10683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:54121: BlockingRecv error: recv got EOF from 127.31.62.130:54121 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Sep 25 10:55:27 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:27 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:27.783437 8819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:27.774172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:59859)
0925 10:55:27.774586 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:27.774602 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:27.774710 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:27.775240 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:27.775244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:27.775643 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:27.775796 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:27.775805 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:27.776996 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:27.777000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:27.778065 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:27.778079 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:27.778905 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:27.778910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:27.779073 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:27.779857 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:27.779875 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:27.783182 (+ 3307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":211,"threads_started":1}
W20260925 10:55:27.783694 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:27.774213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:59859 (local address 127.31.62.190:36505)
0925 10:55:27.774415 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:27.774421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:27.774677 (+ 256us) server_negotiation.cc:407] Connection header received
0925 10:55:27.774854 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:27.774859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:27.774959 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:27.775135 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:27.775953 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:27.776818 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:27.778288 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:27.778726 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:27.778835 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:27.783419 (+ 4584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:59859: BlockingRecv error: recv got EOF from 127.31.62.131:59859 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 25 10:55:28 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:28 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:28.017759 8820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:28.009107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:60247)
0925 10:55:28.009614 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:28.009629 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:28.009764 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:28.010192 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:28.010197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:28.010632 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:28.010789 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:28.010799 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:28.011805 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:28.011810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:28.012766 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:28.012780 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:28.013446 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:28.013452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:28.013614 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:28.014183 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:28.014201 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:28.017457 (+ 3256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":266,"threads_started":1}
W20260925 10:55:28.017928 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:28.009203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:60247 (local address 127.31.62.190:36505)
0925 10:55:28.009389 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:28.009396 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:28.009717 (+ 321us) server_negotiation.cc:407] Connection header received
0925 10:55:28.009901 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:28.009905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:28.009968 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:28.010057 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:28.010930 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:28.011658 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:28.012924 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:28.013308 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:28.013415 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:28.017724 (+ 4309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:60247: BlockingRecv error: recv got EOF from 127.31.62.129:60247 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 25 10:55:28 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:28 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:28.455564 8821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:28.445777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:46207)
0925 10:55:28.446336 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:28.446357 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:28.446477 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:28.446916 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:28.446920 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:28.447363 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:28.447525 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:28.447534 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:28.448620 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:28.448624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:28.449639 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:28.449653 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:28.450505 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:28.450510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:28.450680 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:28.451324 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:28.451348 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:28.455316 (+ 3968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":321,"threads_started":1}
W20260925 10:55:28.455821 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:28.445992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:46207 (local address 127.31.62.190:36505)
0925 10:55:28.446242 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:28.446250 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:28.446440 (+ 190us) server_negotiation.cc:407] Connection header received
0925 10:55:28.446664 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:28.446669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:28.446738 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:28.446828 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:28.447689 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:28.448469 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:28.449804 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:28.450324 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:28.450431 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:28.455529 (+ 5098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:46207: BlockingRecv error: recv got EOF from 127.31.62.130:46207 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Sep 25 10:55:28 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:28 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:28.793897 8822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:28.784500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:57531)
0925 10:55:28.785215 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:28.785233 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:28.785463 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:28.785889 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:28.785894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:28.786328 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:28.786469 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:28.786478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:28.787448 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:28.787452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:28.788391 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:28.788403 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:28.789267 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:28.789274 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:28.789463 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:28.790082 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:28.790100 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:28.793510 (+ 3410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":399,"threads_started":1}
W20260925 10:55:28.794000 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:28.784663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:57531 (local address 127.31.62.190:36505)
0925 10:55:28.784962 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:28.784968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:28.785343 (+ 375us) server_negotiation.cc:407] Connection header received
0925 10:55:28.785655 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:28.785660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:28.785741 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:28.785833 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:28.786620 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:28.787314 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:28.788533 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:28.789034 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:28.789163 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:28.793779 (+ 4616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:57531: BlockingRecv error: recv got EOF from 127.31.62.131:57531 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Sep 25 10:55:29 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:29 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:29.028057 8823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:29.018714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:51971)
0925 10:55:29.019167 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:29.019187 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:29.019302 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:29.019690 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:29.019694 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:29.020087 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:29.020247 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:29.020255 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:29.021385 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:29.021390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:29.022387 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:29.022401 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:29.023197 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:29.023202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:29.023333 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:29.023955 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:29.023973 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:29.027797 (+ 3824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":244,"threads_started":1}
W20260925 10:55:29.028443 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:29.018811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:51971 (local address 127.31.62.190:36505)
0925 10:55:29.018986 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:29.018990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:29.019269 (+ 279us) server_negotiation.cc:407] Connection header received
0925 10:55:29.019441 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:29.019445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:29.019522 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:29.019607 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:29.020381 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:29.021225 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:29.022564 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:29.022989 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:29.023098 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:29.028215 (+ 5117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:51971: BlockingRecv error: recv got EOF from 127.31.62.129:51971 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 25 10:55:29 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:29 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:29.465636 8824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:29.456813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:34009)
0925 10:55:29.457357 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:29.457383 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:29.457516 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:29.457838 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:29.457842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:29.458234 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:29.458404 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:29.458412 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:29.459374 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:29.459379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:29.460440 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:29.460455 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:29.461198 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:29.461204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:29.461329 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:29.461889 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:29.461909 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:29.465389 (+ 3480us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":308,"threads_started":1}
W20260925 10:55:29.465822 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:29.456936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:34009 (local address 127.31.62.190:36505)
0925 10:55:29.457182 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:29.457186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:29.457480 (+ 294us) server_negotiation.cc:407] Connection header received
0925 10:55:29.457612 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:29.457615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:29.457716 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:29.457806 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:29.458530 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:29.459234 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:29.460573 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:29.461055 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:29.461152 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:29.465603 (+ 4451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:34009: BlockingRecv error: recv got EOF from 127.31.62.130:34009 (error 108)
Metrics: {"server-negotiator.queue_time_us":135}
Sep 25 10:55:29 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:29 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:29.803740 8825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:29.795024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:38817)
0925 10:55:29.795550 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:29.795579 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:29.795706 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:29.796129 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:29.796133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:29.796512 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:29.796706 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:29.796716 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:29.797755 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:29.797759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:29.798690 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:29.798703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:29.799397 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:29.799402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:29.799536 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:29.800100 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:29.800118 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:29.803412 (+ 3294us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":258,"threads_started":1}
W20260925 10:55:29.803941 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:29.795122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:38817 (local address 127.31.62.190:36505)
0925 10:55:29.795293 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:29.795297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:29.795669 (+ 372us) server_negotiation.cc:407] Connection header received
0925 10:55:29.795803 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:29.795807 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:29.795885 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:29.795975 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:29.796876 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:29.797606 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:29.798824 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:29.799216 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:29.799329 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:29.803614 (+ 4285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:38817: BlockingRecv error: recv got EOF from 127.31.62.131:38817 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 25 10:55:30 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:30 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:30.038537 8826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:30.029257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:54175)
0925 10:55:30.029876 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:30.029896 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:30.030044 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:30.030590 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:30.030593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:30.031012 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:30.031198 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:30.031208 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:30.032254 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:30.032259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:30.033222 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:30.033234 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:30.033930 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:30.033935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:30.034096 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:30.034651 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:30.034668 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:30.038201 (+ 3533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":261,"threads_started":1}
W20260925 10:55:30.038627 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:30.029353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:54175 (local address 127.31.62.190:36505)
0925 10:55:30.029562 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:30.029569 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:30.029980 (+ 411us) server_negotiation.cc:407] Connection header received
0925 10:55:30.030181 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:30.030185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:30.030264 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:30.030360 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:30.031344 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:30.032105 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:30.033367 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:30.033784 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:30.033891 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:30.038435 (+ 4544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:54175: BlockingRecv error: recv got EOF from 127.31.62.129:54175 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 25 10:55:30 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:30 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:30.475798 8827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:30.466488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:58471)
0925 10:55:30.466987 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:30.467006 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:30.467118 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:30.467591 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:30.467594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:30.467984 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:30.468217 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:30.468227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:30.469421 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:30.469425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:30.470406 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:30.470419 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:30.471301 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:30.471305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:30.471415 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:30.472015 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:30.472033 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:30.475524 (+ 3491us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":225,"threads_started":1}
W20260925 10:55:30.476107 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:30.466552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:58471 (local address 127.31.62.190:36505)
0925 10:55:30.466794 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:30.466802 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:30.467101 (+ 299us) server_negotiation.cc:407] Connection header received
0925 10:55:30.467274 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:30.467278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:30.467355 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:30.467443 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:30.468377 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:30.469278 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:30.470564 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:30.471115 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:30.471250 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:30.475817 (+ 4567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:58471: BlockingRecv error: recv got EOF from 127.31.62.130:58471 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Sep 25 10:55:30 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:30 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:30.814177 8828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:30.804676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:37157)
0925 10:55:30.805312 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:30.805328 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:30.805514 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:30.805883 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:30.805888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:30.806331 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:30.806524 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:30.806535 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:30.807593 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:30.807598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:30.808575 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:30.808588 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:30.809424 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:30.809430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:30.809604 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:30.810197 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:30.810216 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:30.813852 (+ 3636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":282,"threads_started":1}
W20260925 10:55:30.814261 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:30.804879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:37157 (local address 127.31.62.190:36505)
0925 10:55:30.805117 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:30.805124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:30.805425 (+ 301us) server_negotiation.cc:407] Connection header received
0925 10:55:30.805649 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:30.805655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:30.805735 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:30.805827 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:30.806694 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:30.807431 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:30.808733 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:30.809251 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:30.809378 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:30.814054 (+ 4676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:37157: BlockingRecv error: recv got EOF from 127.31.62.131:37157 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 25 10:55:31 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:31 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:31.048565 8829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:31.039574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:42177)
0925 10:55:31.040077 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:31.040094 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:31.040218 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:31.040547 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:31.040551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:31.041050 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:31.041239 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:31.041249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:31.042257 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:31.042261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:31.043269 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:31.043281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:31.044092 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:31.044098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:31.044221 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:31.044809 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:31.044828 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:31.048314 (+ 3486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":234,"threads_started":1}
W20260925 10:55:31.048686 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:31.039820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:42177 (local address 127.31.62.190:36505)
0925 10:55:31.040041 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:31.040046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:31.040165 (+ 119us) server_negotiation.cc:407] Connection header received
0925 10:55:31.040343 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:31.040346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:31.040406 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:31.040476 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:31.041374 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:31.042114 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:31.043407 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:31.043826 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:31.043943 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:31.048490 (+ 4547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:42177: BlockingRecv error: recv got EOF from 127.31.62.129:42177 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 25 10:55:31 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:31 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:31.486702 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:31.477143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:48585)
0925 10:55:31.477643 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:31.477666 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:31.477813 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:31.478480 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:31.478484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:31.478976 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:31.479136 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:31.479145 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:31.480406 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:31.480410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:31.481422 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:31.481434 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:31.482147 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:31.482152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:31.482277 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:31.482844 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:31.482862 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:31.486432 (+ 3570us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":243,"threads_started":1}
W20260925 10:55:31.486946 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:31.477193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:48585 (local address 127.31.62.190:36505)
0925 10:55:31.477395 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:31.477401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:31.477751 (+ 350us) server_negotiation.cc:407] Connection header received
0925 10:55:31.478204 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:31.478208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:31.478301 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:31.478443 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:31.479278 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:31.480263 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:31.481558 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:31.481963 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:31.482117 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:31.486680 (+ 4563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:48585: BlockingRecv error: recv got EOF from 127.31.62.130:48585 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 25 10:55:31 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:31 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:31.824807 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:31.815301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:45047)
0925 10:55:31.815974 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:31.815996 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:31.816142 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:31.816523 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:31.816528 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:31.817101 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:31.817273 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:31.817282 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:31.818385 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:31.818389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:31.819414 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:31.819427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:31.820290 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:31.820295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:31.820416 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:31.821049 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:31.821073 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:31.824539 (+ 3466us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":388,"threads_started":1}
W20260925 10:55:31.824993 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:31.815351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:45047 (local address 127.31.62.190:36505)
0925 10:55:31.815508 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:31.815512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:31.816083 (+ 571us) server_negotiation.cc:407] Connection header received
0925 10:55:31.816257 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:31.816263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:31.816363 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:31.816450 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:31.817415 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:31.818249 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:31.819602 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:31.820092 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:31.820201 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:31.824723 (+ 4522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:45047: BlockingRecv error: recv got EOF from 127.31.62.131:45047 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 25 10:55:32 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:32 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:32.058506 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:32.049722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:53767)
0925 10:55:32.050206 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:32.050262 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:32.050403 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:32.050741 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:32.050746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:32.051164 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:32.051314 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:32.051323 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:32.052307 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:32.052313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:32.053319 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:32.053332 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:32.054062 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:32.054068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:32.054237 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:32.054857 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:32.054876 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:32.058187 (+ 3311us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":267,"threads_started":1}
W20260925 10:55:32.058610 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:32.049808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:53767 (local address 127.31.62.190:36505)
0925 10:55:32.050029 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:32.050035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:32.050350 (+ 315us) server_negotiation.cc:407] Connection header received
0925 10:55:32.050532 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:32.050538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:32.050603 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:32.050684 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:32.051455 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:32.052162 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:32.053473 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:32.053904 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:32.054018 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:32.058405 (+ 4387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:53767: BlockingRecv error: recv got EOF from 127.31.62.129:53767 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 25 10:55:32 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:32 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:32.498587 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:32.487726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:51879)
0925 10:55:32.488279 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:32.488305 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:32.488441 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:32.489097 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:32.489103 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:32.489610 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:32.489794 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:32.489808 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:32.491222 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:32.491226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:32.492687 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:32.492704 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:32.493601 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:32.493607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:32.493790 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:32.494508 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:32.494527 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:32.498349 (+ 3822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":247,"threads_started":1}
W20260925 10:55:32.498800 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:32.487934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:51879 (local address 127.31.62.190:36505)
0925 10:55:32.488292 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:32.488299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:32.488391 (+ 92us) server_negotiation.cc:407] Connection header received
0925 10:55:32.488597 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:32.488602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:32.488688 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:32.488787 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:32.489936 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:32.491068 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:32.492831 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:32.493432 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:32.493561 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:32.498565 (+ 5004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:51879: BlockingRecv error: recv got EOF from 127.31.62.130:51879 (error 108)
Metrics: {"server-negotiator.queue_time_us":244}
Sep 25 10:55:32 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:32 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:32.835229 8834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:32.825975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:40673)
0925 10:55:32.826573 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:32.826594 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:32.826744 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:32.827048 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:32.827051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:32.827426 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:32.827575 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:32.827583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:32.828700 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:32.828704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:32.829770 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:32.829783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:32.830673 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:32.830679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:32.830832 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:32.831418 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:32.831437 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:32.834916 (+ 3479us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":310,"threads_started":1}
W20260925 10:55:32.835296 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:32.826015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:40673 (local address 127.31.62.190:36505)
0925 10:55:32.826233 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:32.826240 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:32.826693 (+ 453us) server_negotiation.cc:407] Connection header received
0925 10:55:32.826847 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:32.826852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:32.826936 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:32.827021 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:32.827739 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:32.828546 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:32.829919 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:32.830445 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:32.830602 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:32.835117 (+ 4515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:40673: BlockingRecv error: recv got EOF from 127.31.62.131:40673 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 25 10:55:33 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:33 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:33.068977 8835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:33.059520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:52579)
0925 10:55:33.060277 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:33.060294 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:33.060425 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:33.060765 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:33.060770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:33.061185 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:33.061342 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:33.061352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:33.062329 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:33.062333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:33.063761 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:33.063778 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:33.064468 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:33.064474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:33.064635 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:33.065282 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:33.065301 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:33.068682 (+ 3381us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":467,"threads_started":1}
W20260925 10:55:33.069140 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:33.059605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:52579 (local address 127.31.62.190:36505)
0925 10:55:33.059821 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:33.059828 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:33.060378 (+ 550us) server_negotiation.cc:407] Connection header received
0925 10:55:33.060561 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:33.060565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:33.060623 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:33.060699 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:33.061486 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:33.062196 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:33.063900 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:33.064324 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:33.064432 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:33.068936 (+ 4504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:52579: BlockingRecv error: recv got EOF from 127.31.62.129:52579 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 25 10:55:33 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:33 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:33.508692 8836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:33.499578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:37317)
0925 10:55:33.500076 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:33.500095 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:33.500225 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:33.500525 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:33.500529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:33.500956 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:33.501134 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:33.501144 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:33.502179 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:33.502184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:33.503184 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:33.503196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:33.503877 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:33.503883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:33.504033 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:33.504628 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:33.504645 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:33.508379 (+ 3734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":276,"threads_started":1}
W20260925 10:55:33.508803 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:33.499648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:37317 (local address 127.31.62.190:36505)
0925 10:55:33.499859 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:33.499866 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:33.500175 (+ 309us) server_negotiation.cc:407] Connection header received
0925 10:55:33.500333 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:33.500339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:33.500403 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:33.500480 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:33.501316 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:33.502050 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:33.503326 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:33.503731 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:33.503850 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:33.508562 (+ 4712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:37317: BlockingRecv error: recv got EOF from 127.31.62.130:37317 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Sep 25 10:55:33 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:33 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:33.845840 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:33.836268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:44249)
0925 10:55:33.836746 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:33.836765 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:33.836928 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:33.837336 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:33.837340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:33.837686 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:33.837846 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:33.837854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:33.838883 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:33.838887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:33.840242 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:33.840257 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:33.841262 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:33.841270 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:33.841443 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:33.842042 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:33.842060 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:33.845491 (+ 3431us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":238,"threads_started":1}
W20260925 10:55:33.845966 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:33.836488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:44249 (local address 127.31.62.190:36505)
0925 10:55:33.836698 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:33.836705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:33.836872 (+ 167us) server_negotiation.cc:407] Connection header received
0925 10:55:33.837068 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:33.837073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:33.837154 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:33.837304 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:33.837977 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:33.838725 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:33.840397 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:33.841033 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:33.841160 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:33.845716 (+ 4556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:44249: BlockingRecv error: recv got EOF from 127.31.62.131:44249 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Sep 25 10:55:34 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:34 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:34.078684 8838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:34.069979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:60071)
0925 10:55:34.070393 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:34.070409 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:34.070535 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:34.070862 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:34.070866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:34.071255 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:34.071421 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:34.071429 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:34.072440 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:34.072445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:34.073553 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:34.073568 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:34.074448 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:34.074452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:34.074576 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:34.075132 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:34.075150 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:34.078406 (+ 3256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":249,"threads_started":1}
W20260925 10:55:34.078783 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:34.070063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:60071 (local address 127.31.62.190:36505)
0925 10:55:34.070265 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:34.070271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:34.070488 (+ 217us) server_negotiation.cc:407] Connection header received
0925 10:55:34.070640 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:34.070643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:34.070705 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:34.070791 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:34.071532 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:34.072290 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:34.073715 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:34.074304 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:34.074413 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:34.078581 (+ 4168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:60071: BlockingRecv error: recv got EOF from 127.31.62.129:60071 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 25 10:55:34 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:34 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:34.518886 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:34.509686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:57553)
0925 10:55:34.510148 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:34.510170 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:34.510286 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:34.510587 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:34.510590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:34.510961 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:34.511189 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:34.511198 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:34.512238 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:34.512242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:34.513173 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:34.513184 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:34.513879 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:34.513883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:34.514000 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:34.514593 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:34.514610 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:34.518562 (+ 3952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":255,"threads_started":1}
W20260925 10:55:34.518983 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:34.509798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:57553 (local address 127.31.62.190:36505)
0925 10:55:34.509953 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:34.509957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:34.510275 (+ 318us) server_negotiation.cc:407] Connection header received
0925 10:55:34.510390 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:34.510394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:34.510465 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:34.510562 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:34.511328 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:34.512103 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:34.513324 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:34.513731 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:34.513830 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:34.518758 (+ 4928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:57553: BlockingRecv error: recv got EOF from 127.31.62.130:57553 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 25 10:55:34 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:34 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:34.856619 8840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:34.846899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:33293)
0925 10:55:34.847444 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:34.847461 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:34.847595 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:34.847969 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:34.847973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:34.848389 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:34.848579 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:34.848588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:34.849677 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:34.849682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:34.850845 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:34.850861 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:34.851841 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:34.851846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:34.852143 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:34.852818 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:34.852842 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:34.856401 (+ 3559us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":309,"threads_started":1}
W20260925 10:55:34.856889 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:34.847036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:33293 (local address 127.31.62.190:36505)
0925 10:55:34.847217 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:34.847224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:34.847563 (+ 339us) server_negotiation.cc:407] Connection header received
0925 10:55:34.847692 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:34.847696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:34.847780 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:34.847872 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:34.848761 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:34.849524 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:34.851005 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:34.851661 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:34.851786 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:34.856633 (+ 4847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:33293: BlockingRecv error: recv got EOF from 127.31.62.131:33293 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 25 10:55:35 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:35 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:35.088723 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:35.079688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:52765)
0925 10:55:35.080251 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:35.080267 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:35.080386 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:35.080734 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:35.080738 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:35.081170 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:35.081376 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:35.081386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:35.082415 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:35.082419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:35.083342 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:35.083352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:35.084139 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:35.084143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:35.084306 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:35.084978 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:35.084996 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:35.088438 (+ 3442us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":269,"threads_started":1}
W20260925 10:55:35.088901 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:35.079792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:52765 (local address 127.31.62.190:36505)
0925 10:55:35.079946 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:35.079951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:35.080372 (+ 421us) server_negotiation.cc:407] Connection header received
0925 10:55:35.080530 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:35.080535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:35.080605 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:35.080703 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:35.081521 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:35.082296 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:35.083500 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:35.083988 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:35.084091 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:35.088665 (+ 4574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:52765: BlockingRecv error: recv got EOF from 127.31.62.129:52765 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 25 10:55:35 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:35 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:35.528642 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:35.519993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:41839)
0925 10:55:35.520756 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:35.520776 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:35.520934 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:35.521244 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:35.521248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:35.521656 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:35.521813 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:35.521821 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:35.522811 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:35.522817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:35.523722 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:35.523733 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:35.524426 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:35.524433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:35.524587 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:35.525246 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:35.525263 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:35.528402 (+ 3139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":519,"threads_started":1}
W20260925 10:55:35.529008 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:35.520286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:41839 (local address 127.31.62.190:36505)
0925 10:55:35.520490 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:35.520498 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:35.520884 (+ 386us) server_negotiation.cc:407] Connection header received
0925 10:55:35.521050 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:35.521054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:35.521117 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:35.521196 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:35.521962 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:35.522684 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:35.523857 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:35.524254 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:35.524366 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:35.528664 (+ 4298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:41839: BlockingRecv error: recv got EOF from 127.31.62.130:41839 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 25 10:55:35 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:35 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:35.866637 8843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:35.857558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:45591)
0925 10:55:35.858075 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:35.858092 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:35.858205 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:35.858570 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:35.858574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:35.858957 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:35.859135 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:35.859144 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:35.860263 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:35.860268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:35.861238 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:35.861251 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:35.861908 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:35.861912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:35.862041 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:35.862627 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:35.862643 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:35.866354 (+ 3711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":279,"threads_started":1}
W20260925 10:55:35.866895 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:35.857660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:45591 (local address 127.31.62.190:36505)
0925 10:55:35.857851 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:35.857856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:35.858170 (+ 314us) server_negotiation.cc:407] Connection header received
0925 10:55:35.858347 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:35.858352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:35.858414 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:35.858499 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:35.859303 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:35.860106 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:35.861370 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:35.861784 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:35.861879 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:35.866646 (+ 4767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:45591: BlockingRecv error: recv got EOF from 127.31.62.131:45591 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 25 10:55:36 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:36 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:36.098474 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:36.089841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:36433)
0925 10:55:36.090310 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:36.090329 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:36.090478 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:36.090853 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:36.090857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:36.091366 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:36.091543 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:36.091555 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:36.092550 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:36.092556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:36.093524 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:36.093536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:36.094178 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:36.094184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:36.094337 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:36.094880 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:36.094897 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:36.098170 (+ 3273us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":265,"threads_started":1}
W20260925 10:55:36.098517 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:36.089916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:36433 (local address 127.31.62.190:36505)
0925 10:55:36.090097 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:36.090103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:36.090441 (+ 338us) server_negotiation.cc:407] Connection header received
0925 10:55:36.090623 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:36.090628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:36.090713 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:36.090797 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:36.091685 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:36.092416 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:36.093652 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:36.094047 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:36.094154 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:36.098324 (+ 4170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:36433: BlockingRecv error: recv got EOF from 127.31.62.129:36433 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 25 10:55:36 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:36 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:36.538465 8845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:36.529617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:41241)
0925 10:55:36.530082 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:36.530101 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:36.530272 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:36.530594 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:36.530598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:36.531002 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:36.531151 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:36.531160 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:36.532196 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:36.532202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:36.533170 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:36.533183 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:36.533894 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:36.533900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:36.534063 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:36.534634 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:36.534651 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:36.538157 (+ 3506us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":244,"threads_started":1}
W20260925 10:55:36.538550 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:36.529782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:41241 (local address 127.31.62.190:36505)
0925 10:55:36.530008 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:36.530014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:36.530175 (+ 161us) server_negotiation.cc:407] Connection header received
0925 10:55:36.530389 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:36.530392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:36.530451 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:36.530551 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:36.531292 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:36.532044 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:36.533315 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:36.533755 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:36.533863 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:36.538301 (+ 4438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:41241: BlockingRecv error: recv got EOF from 127.31.62.130:41241 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 25 10:55:36 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:36 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.131@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:36.877297 8846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:36.867585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.131:54303)
0925 10:55:36.867999 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:36.868021 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:36.868181 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:36.868499 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:36.868503 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:36.868934 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:36.869154 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:36.869164 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:36.870155 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:36.870159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:36.871059 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:36.871071 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:36.871755 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:36.871759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:36.871939 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:36.872648 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:36.872671 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:36.876887 (+ 4216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":218,"threads_started":1}
W20260925 10:55:36.877357 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:36.867683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:54303 (local address 127.31.62.190:36505)
0925 10:55:36.867910 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:36.867915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:36.868130 (+ 215us) server_negotiation.cc:407] Connection header received
0925 10:55:36.868303 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:36.868306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:36.868369 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:36.868451 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:36.869300 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:36.870009 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:36.871181 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:36.871598 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:36.871694 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:36.877130 (+ 5436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.131:54303: BlockingRecv error: recv got EOF from 127.31.62.131:54303 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Sep 25 10:55:37 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:37 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.129@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:37.108476 8847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:37.099872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.129:56975)
0925 10:55:37.100395 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:37.100410 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:37.100529 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:37.100841 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:37.100870 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:37.101252 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:37.101424 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:37.101433 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:37.102421 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:37.102427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:37.103335 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:37.103347 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:37.104025 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:37.104031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:37.104202 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:37.104791 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:37.104809 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:37.108210 (+ 3401us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":246,"threads_started":1}
W20260925 10:55:37.108799 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:37.099932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:56975 (local address 127.31.62.190:36505)
0925 10:55:37.100260 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:37.100268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:37.100481 (+ 213us) server_negotiation.cc:407] Connection header received
0925 10:55:37.100641 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:37.100646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:37.100706 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:37.100793 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:37.101569 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:37.102280 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:37.103473 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:37.103888 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:37.103994 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:37.108484 (+ 4490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.129:56975: BlockingRecv error: recv got EOF from 127.31.62.129:56975 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
Sep 25 10:55:37 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 10:55:37 dist-test-slave-nx7s krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.62.130@KRBTEST.COM for oryx/127.31.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 10:55:37.553628 8848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:37.539770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:36505 (local address 127.31.62.130:58899)
0925 10:55:37.540218 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:37.540243 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:37.540377 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:37.540731 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:37.540736 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:37.541154 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:37.541335 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:37.541344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:37.542339 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:37.542343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:37.543303 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:37.543314 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:37.543950 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:37.543956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:37.544120 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:37.549735 (+ 5615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:37.549756 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:37.553298 (+ 3542us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.62.190:36505: Server oryx/127.31.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":240,"threads_started":1}
W20260925 10:55:37.553789 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:37.539845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:58899 (local address 127.31.62.190:36505)
0925 10:55:37.540019 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:37.540025 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:37.540329 (+ 304us) server_negotiation.cc:407] Connection header received
0925 10:55:37.540488 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:37.540492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:37.540556 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:37.540650 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:37.541474 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:37.542191 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:37.543433 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:37.543817 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:37.543923 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:37.553574 (+ 9651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.62.130:58899: BlockingRecv error: recv got EOF from 127.31.62.130:58899 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 10:55:37.681685 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 8388
I20260925 10:55:37.694517 8850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:37.682963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55150 (local address 127.31.62.129:44081)
0925 10:55:37.683684 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:37.683693 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:37.683726 (+ 33us) server_negotiation.cc:407] Connection header received
0925 10:55:37.683809 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:37.683823 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:37.684107 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:37.684336 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:37.685825 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:37.686980 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:37.688267 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:37.688825 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:37.688970 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:37.691360 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:37.691370 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:37.691384 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:37.691434 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:37.693981 (+ 2547us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.62.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0925 10:55:37.694281 (+ 300us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55150: Request ticket server kudu/127.31.62.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":403,"threads_started":1}
W20260925 10:55:37.694622 8850 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55150: Request ticket server kudu/127.31.62.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260925 10:55:37.694706 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 8388: Not authorized: Client connection negotiation failed: client connection to 127.31.62.129:44081: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.62.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260925 10:55:37.694904 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 8388
I20260925 10:55:37.708633 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 8539
I20260925 10:55:37.721604 8851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:37.709911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59062 (local address 127.31.62.130:37209)
0925 10:55:37.710594 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:37.710601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:37.710628 (+ 27us) server_negotiation.cc:407] Connection header received
0925 10:55:37.710689 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:37.710695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:37.710927 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:37.711095 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:37.712581 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:37.713790 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:37.715053 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:37.715647 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:37.715745 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:37.718552 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:37.718562 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:37.718576 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:37.718621 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:37.721079 (+ 2458us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.62.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0925 10:55:37.721360 (+ 281us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59062: Request ticket server kudu/127.31.62.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":369,"threads_started":1}
W20260925 10:55:37.721699 8851 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59062: Request ticket server kudu/127.31.62.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260925 10:55:37.721741 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 8539: Not authorized: Client connection negotiation failed: client connection to 127.31.62.130:37209: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.62.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260925 10:55:37.721904 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 8539
I20260925 10:55:37.734939 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 8672
I20260925 10:55:37.747495 8852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:37.736130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44050 (local address 127.31.62.131:39853)
0925 10:55:37.736735 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:37.736742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:37.736770 (+ 28us) server_negotiation.cc:407] Connection header received
0925 10:55:37.736843 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:37.736970 (+ 127us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:37.737273 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:37.737454 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:37.739175 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:37.740274 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:37.741485 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:37.742051 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:37.742157 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:37.744533 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:37.744544 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:37.744563 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:37.744605 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:37.746960 (+ 2355us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.62.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0925 10:55:37.747243 (+ 283us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44050: Request ticket server kudu/127.31.62.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":295,"threads_started":1}
W20260925 10:55:37.747601 8852 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44050: Request ticket server kudu/127.31.62.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260925 10:55:37.747687 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 8672: Not authorized: Client connection negotiation failed: client connection to 127.31.62.131:39853: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.31.62.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260925 10:55:37.747833 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 8672
I20260925 10:55:37.760875 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 8319
I20260925 10:55:37.762152 8381 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:37.875186 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 8319
2026-09-25T10:55:37Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (20998 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-25T10:55:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:55:37Z Disabled control of system clock
I20260925 10:55:37.945317 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:45045
--webserver_interface=127.31.62.190
--webserver_port=0
--builtin_ntp_servers=127.31.62.148:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:45045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 10:55:38.273312 8857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:38.273617 8857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:38.273694 8857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:38.281133 8857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:55:38.281258 8857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:38.281293 8857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:38.281348 8857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:55:38.281390 8857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:55:38.290856 8857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:45045
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:45045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:38.292138 8857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:38.293963 8857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:38.305904 8862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:38.305987 8865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:38.306005 8863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:38.306700 8857 server_base.cc:1034] running on GCE node
I20260925 10:55:38.307546 8857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:38.309074 8857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:38.310297 8857 hybrid_clock.cc:648] HybridClock initialized: now 1790333738310249 us; error 43 us; skew 500 ppm
I20260925 10:55:38.310640 8857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:55:38.312979 8857 webserver.cc:460] Webserver started at http://127.31.62.190:42573/ using document root <none> and password file <none>
I20260925 10:55:38.313673 8857 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:38.313774 8857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:38.314081 8857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:38.317051 8857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "735c8e4ed96d4fdb90eb507665c97db3"
format_stamp: "Formatted at 2026-09-25 10:55:38 on dist-test-slave-nx7s"
server_key: "f4be76d72d741ce9447376013a5493f1"
server_key_iv: "610f8e7186acc56860ffbfb450fa870e"
server_key_version: "encryptionkey@0"
I20260925 10:55:38.317834 8857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "735c8e4ed96d4fdb90eb507665c97db3"
format_stamp: "Formatted at 2026-09-25 10:55:38 on dist-test-slave-nx7s"
server_key: "f4be76d72d741ce9447376013a5493f1"
server_key_iv: "610f8e7186acc56860ffbfb450fa870e"
server_key_version: "encryptionkey@0"
I20260925 10:55:38.323181 8857 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:55:38.327308 8871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:38.329097 8857 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 10:55:38.329303 8857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "735c8e4ed96d4fdb90eb507665c97db3"
format_stamp: "Formatted at 2026-09-25 10:55:38 on dist-test-slave-nx7s"
server_key: "f4be76d72d741ce9447376013a5493f1"
server_key_iv: "610f8e7186acc56860ffbfb450fa870e"
server_key_version: "encryptionkey@0"
I20260925 10:55:38.329531 8857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:38.354242 8857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:38.355225 8857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:38.355475 8857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:38.375434 8857 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:45045
I20260925 10:55:38.375451 8922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:45045 every 8 connection(s)
I20260925 10:55:38.376829 8857 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:55:38.377401 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 8857
I20260925 10:55:38.377626 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:55:38.378347 31994 external_mini_cluster.cc:1428] Setting key de945cfd075e36c36e595c2b107eb9db
I20260925 10:55:38.381697 8923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:38.395332 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:38.380381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42676 (local address 127.31.62.190:45045)
0925 10:55:38.381457 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:38.381473 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:38.381533 (+ 60us) server_negotiation.cc:407] Connection header received
0925 10:55:38.383842 (+ 2309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:38.383874 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:38.384590 (+ 716us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:38.385196 (+ 606us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:38.385847 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:38.387929 (+ 2082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:38.389488 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:38.390565 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:38.390763 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:38.392045 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:38.392057 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:38.392080 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:55:38.392926 (+ 846us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:38.393038 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:55:38.393073 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:38.393194 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:38.393476 (+ 282us) server_negotiation.cc:299] Negotiation successful
0925 10:55:38.393889 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":308,"threads_started":1}
I20260925 10:55:38.396436 8923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3: Bootstrap starting.
I20260925 10:55:38.399791 8923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:38.400931 8923 log.cc:824] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:38.404045 8923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3: No bootstrap required, opened a new log
I20260925 10:55:38.411876 8923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735c8e4ed96d4fdb90eb507665c97db3" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45045 } }
I20260925 10:55:38.412410 8923 raft_consensus.cc:374] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:38.412529 8923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 735c8e4ed96d4fdb90eb507665c97db3, State: Initialized, Role: FOLLOWER
I20260925 10:55:38.413206 8923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735c8e4ed96d4fdb90eb507665c97db3" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45045 } }
I20260925 10:55:38.413448 8923 raft_consensus.cc:388] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:55:38.413581 8923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:55:38.413797 8923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:38.416033 8923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735c8e4ed96d4fdb90eb507665c97db3" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45045 } }
I20260925 10:55:38.416574 8923 leader_election.cc:302] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 735c8e4ed96d4fdb90eb507665c97db3; no voters:
I20260925 10:55:38.417201 8923 leader_election.cc:288] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:55:38.417354 8928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:55:38.418002 8928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [term 1 LEADER]: Becoming Leader. State: Replica: 735c8e4ed96d4fdb90eb507665c97db3, State: Running, Role: LEADER
I20260925 10:55:38.418558 8928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735c8e4ed96d4fdb90eb507665c97db3" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45045 } }
I20260925 10:55:38.419248 8923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:55:38.421316 8929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "735c8e4ed96d4fdb90eb507665c97db3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735c8e4ed96d4fdb90eb507665c97db3" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45045 } } }
I20260925 10:55:38.421832 8929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:38.422072 8930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 735c8e4ed96d4fdb90eb507665c97db3. Latest consensus state: current_term: 1 leader_uuid: "735c8e4ed96d4fdb90eb507665c97db3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735c8e4ed96d4fdb90eb507665c97db3" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 45045 } } }
I20260925 10:55:38.422397 8930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:38.424103 8937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:55:38.429762 8937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:55:38.440160 8937 catalog_manager.cc:1347] Generated new cluster ID: fe95332c3ff64335919542b94056f209
I20260925 10:55:38.440300 8937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:55:38.454402 8937 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:55:38.456373 8937 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:55:38.470425 8937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 735c8e4ed96d4fdb90eb507665c97db3: Generated new TSK 0
I20260925 10:55:38.471390 8937 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:55:38.485562 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:45045
--builtin_ntp_servers=127.31.62.148:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 10:55:38.811327 8947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:38.811616 8947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:38.811738 8947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:38.818982 8947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:38.819089 8947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:38.819226 8947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:55:38.828773 8947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:45045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:38.830140 8947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:38.831897 8947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:38.844893 8952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:38.844801 8955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:38.845160 8953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:38.846079 8947 server_base.cc:1034] running on GCE node
I20260925 10:55:38.846750 8947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:38.847766 8947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:38.848986 8947 hybrid_clock.cc:648] HybridClock initialized: now 1790333738848924 us; error 52 us; skew 500 ppm
I20260925 10:55:38.849354 8947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:55:38.851763 8947 webserver.cc:460] Webserver started at http://127.31.62.129:37805/ using document root <none> and password file <none>
I20260925 10:55:38.852510 8947 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:38.852623 8947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:38.853012 8947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:38.855971 8947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "76bb4685ab5f4c09999a1f95bcbad96e"
format_stamp: "Formatted at 2026-09-25 10:55:38 on dist-test-slave-nx7s"
server_key: "610fc9924ef01ddb758d239caae66a15"
server_key_iv: "6618b145c52f60d1eb6890e3686e6c42"
server_key_version: "encryptionkey@0"
I20260925 10:55:38.856750 8947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "76bb4685ab5f4c09999a1f95bcbad96e"
format_stamp: "Formatted at 2026-09-25 10:55:38 on dist-test-slave-nx7s"
server_key: "610fc9924ef01ddb758d239caae66a15"
server_key_iv: "6618b145c52f60d1eb6890e3686e6c42"
server_key_version: "encryptionkey@0"
I20260925 10:55:38.862195 8947 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:55:38.865916 8961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:38.867393 8947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 10:55:38.867573 8947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "76bb4685ab5f4c09999a1f95bcbad96e"
format_stamp: "Formatted at 2026-09-25 10:55:38 on dist-test-slave-nx7s"
server_key: "610fc9924ef01ddb758d239caae66a15"
server_key_iv: "6618b145c52f60d1eb6890e3686e6c42"
server_key_version: "encryptionkey@0"
I20260925 10:55:38.867794 8947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:38.896086 8947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:38.897181 8947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:38.897464 8947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:38.898447 8947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:38.899879 8947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:38.899973 8947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:38.900081 8947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:38.900163 8947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:38.930207 8947 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:44475
I20260925 10:55:38.930282 9073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:44475 every 8 connection(s)
I20260925 10:55:38.931602 8947 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:55:38.940495 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 8947
I20260925 10:55:38.940694 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:55:38.941322 31994 external_mini_cluster.cc:1428] Setting key 4b25e3b864da37f15fa709b680cc403f
I20260925 10:55:38.944002 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:45045
--builtin_ntp_servers=127.31.62.148:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 10:55:38.944824 9076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:38.934402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:54435 (local address 127.31.62.190:45045)
0925 10:55:38.934853 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:38.934860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:38.936240 (+ 1380us) server_negotiation.cc:407] Connection header received
0925 10:55:38.938076 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:38.938081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:38.938156 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:38.938266 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:38.939373 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:38.940107 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:38.941882 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:38.942390 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:38.942527 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:38.943984 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:38.943989 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:38.943992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:55:38.944009 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:38.944048 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:55:38.944061 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:38.944144 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:38.944382 (+ 238us) server_negotiation.cc:299] Negotiation successful
0925 10:55:38.944479 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":230,"threads_started":1}
I20260925 10:55:38.945966 9077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:38.935088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:45045 (local address 127.31.62.129:54435)
0925 10:55:38.936066 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:38.936118 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:38.937783 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:38.938466 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:38.938474 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:38.938610 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:38.939196 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:38.939212 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:38.940287 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:38.940292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:38.941733 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:38.941746 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:38.942676 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:38.942685 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:38.942927 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:38.943662 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:55:38.943665 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:38.943707 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:55:38.943795 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:38.944150 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:38.944162 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:38.944170 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 10:55:38.944368 (+ 198us) client_negotiation.cc:240] Negotiation successful
0925 10:55:38.944839 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":288,"threads_started":1}
I20260925 10:55:38.947587 9074 heartbeater.cc:347] Connected to a master server at 127.31.62.190:45045
I20260925 10:55:38.947881 9074 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:38.948663 9074 heartbeater.cc:511] Master 127.31.62.190:45045 requested a full tablet report, sending...
I20260925 10:55:38.950894 8888 ts_manager.cc:194] Registered new tserver with Master: 76bb4685ab5f4c09999a1f95bcbad96e (127.31.62.129:44475)
I20260925 10:55:38.952414 8888 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.62.129:54435
W20260925 10:55:39.272266 9078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:39.272563 9078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:39.272650 9078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:39.280084 9078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:39.280301 9078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:39.280442 9078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:55:39.289902 9078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:45045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:39.291275 9078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:39.293238 9078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:39.307480 9083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:39.307301 9084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:39.308120 9086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:39.309162 9078 server_base.cc:1034] running on GCE node
I20260925 10:55:39.309849 9078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:39.310922 9078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:39.312196 9078 hybrid_clock.cc:648] HybridClock initialized: now 1790333739312134 us; error 59 us; skew 500 ppm
I20260925 10:55:39.312538 9078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:55:39.315069 9078 webserver.cc:460] Webserver started at http://127.31.62.130:34775/ using document root <none> and password file <none>
I20260925 10:55:39.315805 9078 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:39.315919 9078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:39.316252 9078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:39.319701 9078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "8a444a08462549fd924be9eed4343c41"
format_stamp: "Formatted at 2026-09-25 10:55:39 on dist-test-slave-nx7s"
server_key: "88dd4f4057801e601c756389dabbce20"
server_key_iv: "edacc50287e84690181ece30072c57d4"
server_key_version: "encryptionkey@0"
I20260925 10:55:39.320722 9078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "8a444a08462549fd924be9eed4343c41"
format_stamp: "Formatted at 2026-09-25 10:55:39 on dist-test-slave-nx7s"
server_key: "88dd4f4057801e601c756389dabbce20"
server_key_iv: "edacc50287e84690181ece30072c57d4"
server_key_version: "encryptionkey@0"
I20260925 10:55:39.326851 9078 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260925 10:55:39.331030 9092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:39.332629 9078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:55:39.332830 9078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "8a444a08462549fd924be9eed4343c41"
format_stamp: "Formatted at 2026-09-25 10:55:39 on dist-test-slave-nx7s"
server_key: "88dd4f4057801e601c756389dabbce20"
server_key_iv: "edacc50287e84690181ece30072c57d4"
server_key_version: "encryptionkey@0"
I20260925 10:55:39.333101 9078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:39.351879 9078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:39.352810 9078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:39.353412 9078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:39.354574 9078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:39.358168 9078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:39.358297 9078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:39.358424 9078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:39.358501 9078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:39.394441 9078 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:34599
I20260925 10:55:39.394481 9204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:34599 every 8 connection(s)
I20260925 10:55:39.395759 9078 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:55:39.398307 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 9078
I20260925 10:55:39.398518 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:55:39.399047 31994 external_mini_cluster.cc:1428] Setting key a2f7656a7daa344a365f49a3f091e40a
I20260925 10:55:39.402092 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:45045
--builtin_ntp_servers=127.31.62.148:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 10:55:39.409034 9076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:39.398344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:42441 (local address 127.31.62.190:45045)
0925 10:55:39.398505 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:39.398509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:39.400492 (+ 1983us) server_negotiation.cc:407] Connection header received
0925 10:55:39.402507 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:39.402512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:39.402591 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:39.402677 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:39.403861 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:39.404577 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:39.406221 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:39.406604 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:39.406712 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:39.408163 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:39.408168 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:39.408171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:55:39.408191 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:39.408229 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:55:39.408242 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:39.408318 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:39.408574 (+ 256us) server_negotiation.cc:299] Negotiation successful
0925 10:55:39.408676 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 10:55:39.410064 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:39.399165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:45045 (local address 127.31.62.130:42441)
0925 10:55:39.400322 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:39.400372 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:39.402220 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:39.402959 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:39.402968 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:39.403080 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:39.403696 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:39.403712 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:39.404741 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:39.404745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:39.406062 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:39.406079 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:39.406830 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:39.406838 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:39.407057 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:39.407770 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:55:39.407774 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:39.407830 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:55:39.407939 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:39.408339 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:39.408350 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:39.408359 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 10:55:39.408540 (+ 181us) client_negotiation.cc:240] Negotiation successful
0925 10:55:39.408938 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":326,"threads_started":1}
I20260925 10:55:39.411544 9205 heartbeater.cc:347] Connected to a master server at 127.31.62.190:45045
I20260925 10:55:39.411862 9205 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:39.412652 9205 heartbeater.cc:511] Master 127.31.62.190:45045 requested a full tablet report, sending...
I20260925 10:55:39.414124 8888 ts_manager.cc:194] Registered new tserver with Master: 8a444a08462549fd924be9eed4343c41 (127.31.62.130:34599)
I20260925 10:55:39.414978 8888 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.62.130:42441
W20260925 10:55:39.728648 9208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:39.728960 9208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:39.729070 9208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:39.736316 9208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:39.736420 9208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:39.736555 9208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:55:39.745605 9208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:45045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:39.746882 9208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:39.748615 9208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:39.761955 9213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:39.761967 9214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:39.762737 9216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:39.762810 9208 server_base.cc:1034] running on GCE node
I20260925 10:55:39.763507 9208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:39.764487 9208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:39.765693 9208 hybrid_clock.cc:648] HybridClock initialized: now 1790333739765637 us; error 47 us; skew 500 ppm
I20260925 10:55:39.766042 9208 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:55:39.768474 9208 webserver.cc:460] Webserver started at http://127.31.62.131:44081/ using document root <none> and password file <none>
I20260925 10:55:39.769243 9208 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:39.769333 9208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:39.769644 9208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:39.772503 9208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "f0ac3ef76d6944ca8948e3f69b876e2e"
format_stamp: "Formatted at 2026-09-25 10:55:39 on dist-test-slave-nx7s"
server_key: "ebcdddbd585aca1dd6449772d4d5859f"
server_key_iv: "e699dcd03d1be264dde1f7dc75798624"
server_key_version: "encryptionkey@0"
I20260925 10:55:39.773307 9208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "f0ac3ef76d6944ca8948e3f69b876e2e"
format_stamp: "Formatted at 2026-09-25 10:55:39 on dist-test-slave-nx7s"
server_key: "ebcdddbd585aca1dd6449772d4d5859f"
server_key_iv: "e699dcd03d1be264dde1f7dc75798624"
server_key_version: "encryptionkey@0"
I20260925 10:55:39.778887 9208 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:55:39.782970 9222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:39.784602 9208 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:55:39.784835 9208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "f0ac3ef76d6944ca8948e3f69b876e2e"
format_stamp: "Formatted at 2026-09-25 10:55:39 on dist-test-slave-nx7s"
server_key: "ebcdddbd585aca1dd6449772d4d5859f"
server_key_iv: "e699dcd03d1be264dde1f7dc75798624"
server_key_version: "encryptionkey@0"
I20260925 10:55:39.785110 9208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:39.808979 9208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:39.809904 9208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:39.810209 9208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:39.811184 9208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:39.812732 9208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:39.812827 9208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:39.812959 9208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:39.813028 9208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:39.844473 9208 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:40909
I20260925 10:55:39.844602 9334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:40909 every 8 connection(s)
I20260925 10:55:39.845850 9208 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:55:39.855216 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 9208
I20260925 10:55:39.855404 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:55:39.855980 31994 external_mini_cluster.cc:1428] Setting key c1e7f7977270e037fc6ebd58feffafb5
I20260925 10:55:39.859210 9076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:39.848399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:58559 (local address 127.31.62.190:45045)
0925 10:55:39.848617 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:39.848623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:39.850099 (+ 1476us) server_negotiation.cc:407] Connection header received
0925 10:55:39.851824 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:39.851829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:39.851895 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:39.851983 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:39.853120 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:39.853868 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:39.856003 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:39.856538 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:39.856674 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:39.858379 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:39.858383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:39.858387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:55:39.858405 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:39.858436 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:55:39.858447 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:39.858520 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:39.858758 (+ 238us) server_negotiation.cc:299] Negotiation successful
0925 10:55:39.858853 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260925 10:55:39.860293 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:39.849048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:45045 (local address 127.31.62.131:58559)
0925 10:55:39.849945 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:39.849981 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:39.851568 (+ 1587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:39.852168 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:39.852176 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:39.852270 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:39.852900 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:39.852920 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:39.854049 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:39.854054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:39.855811 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:39.855827 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:39.856807 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:39.856815 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:39.857178 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:39.857946 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:55:39.857949 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:39.858008 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:55:39.858198 (+ 190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:39.858532 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:39.858543 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:39.858551 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 10:55:39.858719 (+ 168us) client_negotiation.cc:240] Negotiation successful
0925 10:55:39.859159 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":268,"threads_started":1}
I20260925 10:55:39.861670 9335 heartbeater.cc:347] Connected to a master server at 127.31.62.190:45045
I20260925 10:55:39.861943 9335 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:39.862713 9335 heartbeater.cc:511] Master 127.31.62.190:45045 requested a full tablet report, sending...
I20260925 10:55:39.864110 8888 ts_manager.cc:194] Registered new tserver with Master: f0ac3ef76d6944ca8948e3f69b876e2e (127.31.62.131:40909)
I20260925 10:55:39.864940 8888 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.62.131:58559
I20260925 10:55:39.871901 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 10:55:39.878643 9076 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:39.876823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42686 (local address 127.31.62.190:45045)
0925 10:55:39.877037 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:39.877042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:39.877535 (+ 493us) server_negotiation.cc:407] Connection header received
0925 10:55:39.877605 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:39.877610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:39.877689 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:39.877782 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:39.878366 (+ 584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42686: BlockingRecv error: recv got EOF from 127.0.0.1:42686 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260925 10:55:39.879228 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 8947
I20260925 10:55:39.887116 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:39.880315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33738 (local address 127.31.62.129:44475)
0925 10:55:39.880835 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:39.880842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:39.880905 (+ 63us) server_negotiation.cc:407] Connection header received
0925 10:55:39.880973 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:39.880980 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:39.881230 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:39.881364 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:39.881767 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:39.882765 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:39.883999 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:39.884563 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:39.884659 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:39.885727 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:39.885735 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:39.885750 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:55:39.886362 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:39.886422 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:55:39.886443 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:39.886536 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:39.886697 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 10:55:39.886814 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":225,"threads_started":1}
I20260925 10:55:39.888410 9069 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:39.969848 9074 heartbeater.cc:503] Master 127.31.62.190:45045 was elected leader, sending a full tablet report...
I20260925 10:55:39.971016 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 8947
W20260925 10:55:39.984714 8872 connection.cc:535] server connection from 127.31.62.129:54435 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 10:55:39.985034 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 9078
I20260925 10:55:39.993352 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:39.985866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50098 (local address 127.31.62.130:34599)
0925 10:55:39.986464 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:39.986479 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:39.986507 (+ 28us) server_negotiation.cc:407] Connection header received
0925 10:55:39.986570 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:39.986576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:39.986803 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:39.986937 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:39.987398 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:39.988460 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:39.990024 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:39.990637 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:39.990763 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:39.991877 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:39.991886 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:39.991900 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:55:39.992512 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:39.992563 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:55:39.992588 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:39.992687 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:39.992892 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 10:55:39.993036 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":319,"threads_started":1}
I20260925 10:55:39.994719 9200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:40.075407 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 9078
I20260925 10:55:40.088217 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 9208
I20260925 10:55:40.096488 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:40.089272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50122 (local address 127.31.62.131:40909)
0925 10:55:40.089914 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:40.089920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:40.089949 (+ 29us) server_negotiation.cc:407] Connection header received
0925 10:55:40.090005 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:40.090011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:40.090253 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:40.090419 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:40.090832 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:40.091872 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:40.093230 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:40.093821 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:40.093974 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:40.095022 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:40.095031 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:40.095046 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:55:40.095658 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:40.095709 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:55:40.095729 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:40.095833 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:40.095997 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 10:55:40.096127 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":361,"threads_started":1}
I20260925 10:55:40.097872 9330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:40.179343 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 9208
I20260925 10:55:40.192283 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 8857
I20260925 10:55:40.193697 8918 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:40.292915 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 8857
2026-09-25T10:55:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2412 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-25T10:55:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:55:40Z Disabled control of system clock
I20260925 10:55:40.359061 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:38913
--webserver_interface=127.31.62.190
--webserver_port=0
--builtin_ntp_servers=127.31.62.148:38487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:38913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 10:55:40.683223 9354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:40.683528 9354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:40.683619 9354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:40.691124 9354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:55:40.691246 9354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:40.691282 9354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:40.691335 9354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:55:40.691377 9354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:55:40.700706 9354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:38487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:38913
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:38913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:40.701964 9354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:40.703683 9354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:40.715440 9359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:40.715478 9362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:40.715535 9360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:40.716125 9354 server_base.cc:1034] running on GCE node
I20260925 10:55:40.716995 9354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:40.718490 9354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:40.719699 9354 hybrid_clock.cc:648] HybridClock initialized: now 1790333740719668 us; error 35 us; skew 500 ppm
I20260925 10:55:40.720101 9354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:55:40.722402 9354 webserver.cc:460] Webserver started at http://127.31.62.190:45737/ using document root <none> and password file <none>
I20260925 10:55:40.723102 9354 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:40.723199 9354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:40.723501 9354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:40.728041 9354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "5e0b8f151aba4ca68f751a15582146f0"
format_stamp: "Formatted at 2026-09-25 10:55:40 on dist-test-slave-nx7s"
server_key: "41397beae03511e3f0855685e2fe5b49"
server_key_iv: "1ddeb4fecd9421df5243013b3ca9dc47"
server_key_version: "encryptionkey@0"
I20260925 10:55:40.729022 9354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "5e0b8f151aba4ca68f751a15582146f0"
format_stamp: "Formatted at 2026-09-25 10:55:40 on dist-test-slave-nx7s"
server_key: "41397beae03511e3f0855685e2fe5b49"
server_key_iv: "1ddeb4fecd9421df5243013b3ca9dc47"
server_key_version: "encryptionkey@0"
I20260925 10:55:40.734612 9354 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:55:40.738432 9368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:40.739938 9354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:55:40.740121 9354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "5e0b8f151aba4ca68f751a15582146f0"
format_stamp: "Formatted at 2026-09-25 10:55:40 on dist-test-slave-nx7s"
server_key: "41397beae03511e3f0855685e2fe5b49"
server_key_iv: "1ddeb4fecd9421df5243013b3ca9dc47"
server_key_version: "encryptionkey@0"
I20260925 10:55:40.740339 9354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:40.746268 9354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:40.747090 9354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:40.747380 9354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:40.764658 9354 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:38913
I20260925 10:55:40.764680 9419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:38913 every 8 connection(s)
I20260925 10:55:40.766120 9354 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:55:40.770011 9420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:40.771173 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 9354
I20260925 10:55:40.771373 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:55:40.772140 31994 external_mini_cluster.cc:1428] Setting key 6b1351c0ca1f3bc9daaf7cafc8d47163
I20260925 10:55:40.781194 9423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:40.774389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47186 (local address 127.31.62.190:38913)
0925 10:55:40.775477 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:40.775488 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:40.775530 (+ 42us) server_negotiation.cc:407] Connection header received
0925 10:55:40.777201 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:40.777240 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:40.777735 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:40.778134 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:40.779315 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:40.779321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:40.779337 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:55:40.779354 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:40.779423 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:55:40.779444 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:40.779657 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:40.779797 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 10:55:40.780069 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":376,"threads_started":1}
I20260925 10:55:40.783613 9420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0: Bootstrap starting.
I20260925 10:55:40.786911 9420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:40.788125 9420 log.cc:824] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:40.791075 9420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0: No bootstrap required, opened a new log
I20260925 10:55:40.798574 9420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e0b8f151aba4ca68f751a15582146f0" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38913 } }
I20260925 10:55:40.798967 9420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:40.799039 9420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0b8f151aba4ca68f751a15582146f0, State: Initialized, Role: FOLLOWER
I20260925 10:55:40.799665 9420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e0b8f151aba4ca68f751a15582146f0" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38913 } }
I20260925 10:55:40.799875 9420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:55:40.800004 9420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:55:40.800153 9420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:40.802680 9420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e0b8f151aba4ca68f751a15582146f0" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38913 } }
I20260925 10:55:40.803565 9420 leader_election.cc:302] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e0b8f151aba4ca68f751a15582146f0; no voters:
I20260925 10:55:40.804256 9420 leader_election.cc:288] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:55:40.804386 9425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:55:40.805099 9425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [term 1 LEADER]: Becoming Leader. State: Replica: 5e0b8f151aba4ca68f751a15582146f0, State: Running, Role: LEADER
I20260925 10:55:40.805661 9425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e0b8f151aba4ca68f751a15582146f0" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38913 } }
I20260925 10:55:40.806547 9420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:55:40.808499 9427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e0b8f151aba4ca68f751a15582146f0. Latest consensus state: current_term: 1 leader_uuid: "5e0b8f151aba4ca68f751a15582146f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e0b8f151aba4ca68f751a15582146f0" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38913 } } }
I20260925 10:55:40.808997 9427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:40.810298 9426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e0b8f151aba4ca68f751a15582146f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e0b8f151aba4ca68f751a15582146f0" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38913 } } }
I20260925 10:55:40.810788 9426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:40.812386 9434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:55:40.817550 9434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:55:40.826437 9434 catalog_manager.cc:1347] Generated new cluster ID: a688bd225c584264835efb4c04d62b62
I20260925 10:55:40.826560 9434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:55:40.843972 9434 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:55:40.844475 9434 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:55:40.853693 9434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e0b8f151aba4ca68f751a15582146f0: Generated new TSK 0
I20260925 10:55:40.854592 9434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:55:40.866245 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:38913
--builtin_ntp_servers=127.31.62.148:38487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 10:55:41.200968 9444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:41.201273 9444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:41.201380 9444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:41.208940 9444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:41.209064 9444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:41.209203 9444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:55:41.218410 9444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:38487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:38913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:41.219744 9444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:41.221575 9444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:41.237450 9450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:41.237411 9452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:41.237466 9449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:41.238471 9444 server_base.cc:1034] running on GCE node
I20260925 10:55:41.239248 9444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:41.240278 9444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:41.241482 9444 hybrid_clock.cc:648] HybridClock initialized: now 1790333741241419 us; error 52 us; skew 500 ppm
I20260925 10:55:41.241813 9444 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:55:41.244213 9444 webserver.cc:460] Webserver started at http://127.31.62.129:46545/ using document root <none> and password file <none>
I20260925 10:55:41.244940 9444 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:41.245067 9444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:41.245391 9444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:41.248360 9444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "6db6f19126104f179feeb1f4541c2016"
format_stamp: "Formatted at 2026-09-25 10:55:41 on dist-test-slave-nx7s"
server_key: "1647cb816100d59a7723c59dc480395a"
server_key_iv: "e513b3825527a41f06c330dba80990a1"
server_key_version: "encryptionkey@0"
I20260925 10:55:41.249220 9444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "6db6f19126104f179feeb1f4541c2016"
format_stamp: "Formatted at 2026-09-25 10:55:41 on dist-test-slave-nx7s"
server_key: "1647cb816100d59a7723c59dc480395a"
server_key_iv: "e513b3825527a41f06c330dba80990a1"
server_key_version: "encryptionkey@0"
I20260925 10:55:41.254729 9444 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:55:41.258405 9458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:41.259946 9444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:55:41.260102 9444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "6db6f19126104f179feeb1f4541c2016"
format_stamp: "Formatted at 2026-09-25 10:55:41 on dist-test-slave-nx7s"
server_key: "1647cb816100d59a7723c59dc480395a"
server_key_iv: "e513b3825527a41f06c330dba80990a1"
server_key_version: "encryptionkey@0"
I20260925 10:55:41.260303 9444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:41.266268 9444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:41.267115 9444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:41.267349 9444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:41.268312 9444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:41.269852 9444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:41.269938 9444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:41.270020 9444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:41.270074 9444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:41.300572 9444 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:43631
I20260925 10:55:41.300602 9570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:43631 every 8 connection(s)
I20260925 10:55:41.302024 9444 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:55:41.310187 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:41.304605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:38583 (local address 127.31.62.190:38913)
0925 10:55:41.305028 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:41.305032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:41.306293 (+ 1261us) server_negotiation.cc:407] Connection header received
0925 10:55:41.308003 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:41.308008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:41.308102 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:41.308193 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:41.309499 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:41.309506 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:41.309511 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:55:41.309531 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:41.309569 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:55:41.309580 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:41.309679 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:41.309853 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 10:55:41.309917 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":222,"threads_started":1}
I20260925 10:55:41.310832 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 9444
I20260925 10:55:41.311014 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:55:41.311407 9574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:41.305267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:38913 (local address 127.31.62.129:38583)
0925 10:55:41.306136 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:41.306172 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:41.307747 (+ 1575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:41.308439 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:41.308447 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:41.308541 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:41.309175 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:55:41.309178 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:41.309250 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:55:41.309344 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:41.309662 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:41.309671 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:41.309679 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 10:55:41.309923 (+ 244us) client_negotiation.cc:240] Negotiation successful
0925 10:55:41.310184 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":253,"threads_started":1}
I20260925 10:55:41.311777 31994 external_mini_cluster.cc:1428] Setting key 3c6de1ab4b2affb05d09efb7eeaa1370
I20260925 10:55:41.312934 9571 heartbeater.cc:347] Connected to a master server at 127.31.62.190:38913
I20260925 10:55:41.313200 9571 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:41.313817 9571 heartbeater.cc:511] Master 127.31.62.190:38913 requested a full tablet report, sending...
I20260925 10:55:41.314580 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:38913
--builtin_ntp_servers=127.31.62.148:38487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 10:55:41.315892 9385 ts_manager.cc:194] Registered new tserver with Master: 6db6f19126104f179feeb1f4541c2016 (127.31.62.129:43631)
W20260925 10:55:41.644300 9575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:41.644600 9575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:41.644716 9575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:41.652151 9575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:41.652282 9575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:41.652393 9575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:55:41.661639 9575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:38487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:38913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:41.662874 9575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:41.664770 9575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:41.678006 9580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:41.678025 9583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:41.678117 9581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:41.679111 9575 server_base.cc:1034] running on GCE node
I20260925 10:55:41.679716 9575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:41.680712 9575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:41.681948 9575 hybrid_clock.cc:648] HybridClock initialized: now 1790333741681884 us; error 52 us; skew 500 ppm
I20260925 10:55:41.682341 9575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:55:41.684791 9575 webserver.cc:460] Webserver started at http://127.31.62.130:37931/ using document root <none> and password file <none>
I20260925 10:55:41.685554 9575 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:41.685652 9575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:41.685928 9575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:41.688781 9575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "b2e63e812cb74601899c2a1aed8e1ec1"
format_stamp: "Formatted at 2026-09-25 10:55:41 on dist-test-slave-nx7s"
server_key: "40e3a75ca6a6da6b41230ab7d72aca10"
server_key_iv: "5b21328dfbd8de76d3225582024410b3"
server_key_version: "encryptionkey@0"
I20260925 10:55:41.689604 9575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "b2e63e812cb74601899c2a1aed8e1ec1"
format_stamp: "Formatted at 2026-09-25 10:55:41 on dist-test-slave-nx7s"
server_key: "40e3a75ca6a6da6b41230ab7d72aca10"
server_key_iv: "5b21328dfbd8de76d3225582024410b3"
server_key_version: "encryptionkey@0"
I20260925 10:55:41.694732 9575 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:55:41.698239 9589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:41.699637 9575 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:55:41.699795 9575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "b2e63e812cb74601899c2a1aed8e1ec1"
format_stamp: "Formatted at 2026-09-25 10:55:41 on dist-test-slave-nx7s"
server_key: "40e3a75ca6a6da6b41230ab7d72aca10"
server_key_iv: "5b21328dfbd8de76d3225582024410b3"
server_key_version: "encryptionkey@0"
I20260925 10:55:41.699996 9575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:41.705787 9575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:41.706605 9575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:41.706871 9575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:41.707778 9575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:41.709241 9575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:41.709340 9575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:41.709444 9575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:41.709514 9575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:41.738973 9575 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:33565
I20260925 10:55:41.739030 9701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:33565 every 8 connection(s)
I20260925 10:55:41.740284 9575 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:55:41.746084 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 9575
I20260925 10:55:41.746258 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:55:41.746821 31994 external_mini_cluster.cc:1428] Setting key 6ac98d768c8cf0416b09209dfd00e03a
I20260925 10:55:41.748883 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:41.742886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:42523 (local address 127.31.62.190:38913)
0925 10:55:41.743057 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:41.743061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:41.744543 (+ 1482us) server_negotiation.cc:407] Connection header received
0925 10:55:41.746504 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:41.746508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:41.746569 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:41.746657 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:41.748176 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:41.748181 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:41.748185 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:55:41.748201 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:41.748235 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:55:41.748250 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:41.748351 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:41.748510 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 10:55:41.748566 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 10:55:41.749900 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:38913
--builtin_ntp_servers=127.31.62.148:38487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 10:55:41.750113 9704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:41.743449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:38913 (local address 127.31.62.130:42523)
0925 10:55:41.744398 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:41.744444 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:41.746161 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:41.746951 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:41.746965 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:41.747098 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:41.747867 (+ 769us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:55:41.747871 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:41.747922 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:55:41.748017 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:41.748339 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:41.748345 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:41.748351 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 10:55:41.748583 (+ 232us) client_negotiation.cc:240] Negotiation successful
0925 10:55:41.748829 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":237,"threads_started":1}
I20260925 10:55:41.751451 9702 heartbeater.cc:347] Connected to a master server at 127.31.62.190:38913
I20260925 10:55:41.751750 9702 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:41.752357 9702 heartbeater.cc:511] Master 127.31.62.190:38913 requested a full tablet report, sending...
I20260925 10:55:41.753702 9385 ts_manager.cc:194] Registered new tserver with Master: b2e63e812cb74601899c2a1aed8e1ec1 (127.31.62.130:33565)
W20260925 10:55:42.077387 9705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:42.077696 9705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:42.077785 9705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:42.085357 9705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:42.085472 9705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:42.085594 9705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:55:42.094869 9705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:38487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:38913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:42.096115 9705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:42.097878 9705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:42.111495 9710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:42.111521 9711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:42.111620 9713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:42.111959 9705 server_base.cc:1034] running on GCE node
I20260925 10:55:42.112896 9705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:42.113917 9705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:42.115162 9705 hybrid_clock.cc:648] HybridClock initialized: now 1790333742115105 us; error 51 us; skew 500 ppm
I20260925 10:55:42.115504 9705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:55:42.117942 9705 webserver.cc:460] Webserver started at http://127.31.62.131:35307/ using document root <none> and password file <none>
I20260925 10:55:42.118671 9705 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:42.118773 9705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:42.119108 9705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:42.122123 9705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "5f25157762d3497682768117d1d5d0d1"
format_stamp: "Formatted at 2026-09-25 10:55:42 on dist-test-slave-nx7s"
server_key: "8f53b0b90eabec39793276641467df1c"
server_key_iv: "273bffa497dc15ec7dd596508a6a23d1"
server_key_version: "encryptionkey@0"
I20260925 10:55:42.122916 9705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "5f25157762d3497682768117d1d5d0d1"
format_stamp: "Formatted at 2026-09-25 10:55:42 on dist-test-slave-nx7s"
server_key: "8f53b0b90eabec39793276641467df1c"
server_key_iv: "273bffa497dc15ec7dd596508a6a23d1"
server_key_version: "encryptionkey@0"
I20260925 10:55:42.128381 9705 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 10:55:42.132192 9719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:42.133733 9705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:55:42.133898 9705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "5f25157762d3497682768117d1d5d0d1"
format_stamp: "Formatted at 2026-09-25 10:55:42 on dist-test-slave-nx7s"
server_key: "8f53b0b90eabec39793276641467df1c"
server_key_iv: "273bffa497dc15ec7dd596508a6a23d1"
server_key_version: "encryptionkey@0"
I20260925 10:55:42.134104 9705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:42.139834 9705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:42.140640 9705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:42.140930 9705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:42.141861 9705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:42.143252 9705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:42.143356 9705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:42.143458 9705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:42.143527 9705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:42.174052 9705 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:38547
I20260925 10:55:42.174109 9832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:38547 every 8 connection(s)
I20260925 10:55:42.175472 9705 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:55:42.180634 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 9705
I20260925 10:55:42.180811 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:55:42.181360 31994 external_mini_cluster.cc:1428] Setting key a5799a932481c61353185c4e3e4df536
I20260925 10:55:42.184954 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:42.178490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:59167 (local address 127.31.62.190:38913)
0925 10:55:42.178674 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:42.178679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:42.180489 (+ 1810us) server_negotiation.cc:407] Connection header received
0925 10:55:42.182670 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:42.182676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:42.182743 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:42.182831 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:42.184225 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:42.184231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:42.184235 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:55:42.184255 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:42.184292 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:55:42.184308 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:42.184427 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:42.184654 (+ 227us) server_negotiation.cc:299] Negotiation successful
0925 10:55:42.184703 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 10:55:42.185998 9835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:42.179093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:38913 (local address 127.31.62.131:59167)
0925 10:55:42.180261 (+ 1168us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:42.180315 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:42.182348 (+ 2033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:42.183085 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:42.183093 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:42.183207 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:42.183877 (+ 670us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:55:42.183879 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:42.183922 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:55:42.184046 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:42.184409 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:42.184420 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:42.184429 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 10:55:42.184746 (+ 317us) client_negotiation.cc:240] Negotiation successful
0925 10:55:42.185020 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":311,"threads_started":1}
I20260925 10:55:42.187536 9833 heartbeater.cc:347] Connected to a master server at 127.31.62.190:38913
I20260925 10:55:42.187896 9833 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:42.188601 9833 heartbeater.cc:511] Master 127.31.62.190:38913 requested a full tablet report, sending...
I20260925 10:55:42.190148 9385 ts_manager.cc:194] Registered new tserver with Master: 5f25157762d3497682768117d1d5d0d1 (127.31.62.131:38547)
I20260925 10:55:42.197165 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 10:55:42.203286 9573 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 10:55:42.201722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47188 (local address 127.31.62.190:38913)
0925 10:55:42.201879 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:42.201884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:42.202307 (+ 423us) server_negotiation.cc:407] Connection header received
0925 10:55:42.202455 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:42.202460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:42.202513 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:42.202596 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:42.203003 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47188: BlockingRecv error: recv got EOF from 127.0.0.1:47188 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 10:55:42.203779 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 9444
I20260925 10:55:42.207412 9842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:42.204786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.31.62.129:43631)
0925 10:55:42.205306 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:42.205312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:42.205337 (+ 25us) server_negotiation.cc:407] Connection header received
0925 10:55:42.205400 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:42.205407 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:42.205620 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:42.205759 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:42.206722 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:42.206729 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:42.206741 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:55:42.206759 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:42.206804 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:55:42.206817 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:42.206956 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:42.207034 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 10:55:42.207132 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":225,"threads_started":1}
I20260925 10:55:42.208703 9566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:42.290463 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 9444
I20260925 10:55:42.303747 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 9575
I20260925 10:55:42.307320 9844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:42.304657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45936 (local address 127.31.62.130:33565)
0925 10:55:42.305232 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:42.305239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:42.305265 (+ 26us) server_negotiation.cc:407] Connection header received
0925 10:55:42.305327 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:42.305333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:42.305550 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:42.305675 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:42.306595 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:42.306602 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:42.306614 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:55:42.306633 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:42.306676 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:55:42.306693 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:42.306814 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:42.306952 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 10:55:42.307036 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":306,"threads_started":1}
I20260925 10:55:42.308570 9697 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:42.390075 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 9575
I20260925 10:55:42.403106 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 9705
I20260925 10:55:42.407080 9846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:42.404199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38262 (local address 127.31.62.131:38547)
0925 10:55:42.404918 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:42.404926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:42.404958 (+ 32us) server_negotiation.cc:407] Connection header received
0925 10:55:42.405026 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:42.405032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:42.405281 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:42.405417 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:42.406439 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:42.406446 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:42.406458 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:55:42.406475 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:42.406521 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:55:42.406535 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:42.406689 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:42.406743 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 10:55:42.406824 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":347,"threads_started":1}
I20260925 10:55:42.408399 9828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:42.488893 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 9705
I20260925 10:55:42.501664 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 9354
I20260925 10:55:42.503026 9415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:42.623240 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 9354
2026-09-25T10:55:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2330 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:55:42 dist-test-slave-nx7s krb5kdc[9851](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:55:42 dist-test-slave-nx7s krb5kdc[9851](info): set up 2 sockets
Sep 25 10:55:42 dist-test-slave-nx7s krb5kdc[9851](info): commencing operation
krb5kdc: starting...
W20260925 10:55:43.646095 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:55:43 dist-test-slave-nx7s krb5kdc[9851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:55:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:55:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:55:43.877310 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:44105
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:44105
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:55:44.203243 9864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:44.203540 9864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:44.203653 9864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:44.210918 9864 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:55:44.211103 9864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:44.211179 9864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:44.211256 9864 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:55:44.211314 9864 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:55:44.221189 9864 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:44105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:44105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:44.222536 9864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:44.223820 9864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:44.235807 9872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:44.235822 9870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:44.235798 9869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:44.236589 9864 server_base.cc:1034] running on GCE node
I20260925 10:55:44.237540 9864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:44.239084 9864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:44.240345 9864 hybrid_clock.cc:648] HybridClock initialized: now 1790333744240280 us; error 63 us; skew 500 ppm
I20260925 10:55:44.240710 9864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:44 dist-test-slave-nx7s krb5kdc[9851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333744, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:44.243975 9864 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:55:44.245682 9864 webserver.cc:460] Webserver started at http://127.31.62.190:46295/ using document root <none> and password file <none>
I20260925 10:55:44.246402 9864 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:44.246513 9864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:44.246827 9864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:44.250041 9864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "46d55d0ea593472ab38dcb02db5fab96"
format_stamp: "Formatted at 2026-09-25 10:55:44 on dist-test-slave-nx7s"
server_key: "87e8fa61a1cffa4dd0ba364435c2eecf"
server_key_iv: "7580662dbc6c8736f12028cd5fc83ce6"
server_key_version: "encryptionkey@0"
I20260925 10:55:44.250880 9864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "46d55d0ea593472ab38dcb02db5fab96"
format_stamp: "Formatted at 2026-09-25 10:55:44 on dist-test-slave-nx7s"
server_key: "87e8fa61a1cffa4dd0ba364435c2eecf"
server_key_iv: "7580662dbc6c8736f12028cd5fc83ce6"
server_key_version: "encryptionkey@0"
I20260925 10:55:44.256604 9864 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260925 10:55:44.260663 9879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:44.262369 9864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:55:44.262550 9864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "46d55d0ea593472ab38dcb02db5fab96"
format_stamp: "Formatted at 2026-09-25 10:55:44 on dist-test-slave-nx7s"
server_key: "87e8fa61a1cffa4dd0ba364435c2eecf"
server_key_iv: "7580662dbc6c8736f12028cd5fc83ce6"
server_key_version: "encryptionkey@0"
I20260925 10:55:44.262734 9864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:44.279385 9864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:44.283965 9864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:44.284230 9864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:44.301964 9864 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:44105
I20260925 10:55:44.301985 9930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:44105 every 8 connection(s)
I20260925 10:55:44.303422 9864 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:55:44.307376 9931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:44.309247 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 9864
I20260925 10:55:44.309453 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:55:44.310228 31994 external_mini_cluster.cc:1428] Setting key adc2d04b8be5d067fa901c6e1fe8c4e5
I20260925 10:55:44.319811 9931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96: Bootstrap starting.
I20260925 10:55:44.323717 9931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96: Neither blocks nor log segments found. Creating new log.
Sep 25 10:55:44 dist-test-slave-nx7s krb5kdc[9851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:55:44.324954 9931 log.cc:824] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:44.327844 9931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96: No bootstrap required, opened a new log
I20260925 10:55:44.333395 9934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:44.312263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42784 (local address 127.31.62.190:44105)
0925 10:55:44.313076 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:44.313088 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:44.313132 (+ 44us) server_negotiation.cc:407] Connection header received
0925 10:55:44.314774 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:44.314799 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:44.315202 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:44.315530 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:44.317344 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:44.318691 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:44.320432 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:44.321194 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:44.321369 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:44.326087 (+ 4718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:44.326095 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:44.326113 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:44.326156 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:44.329153 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:44.329917 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:44.329924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:44.329930 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:44.330120 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:44.330504 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:44.330508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:44.330510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:44.331073 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:44.331355 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:44.331825 (+ 470us) server_negotiation.cc:299] Negotiation successful
0925 10:55:44.332262 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":375,"threads_started":1}
I20260925 10:55:44.335894 9931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d55d0ea593472ab38dcb02db5fab96" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44105 } }
I20260925 10:55:44.336321 9931 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:44.336433 9931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46d55d0ea593472ab38dcb02db5fab96, State: Initialized, Role: FOLLOWER
I20260925 10:55:44.337059 9931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d55d0ea593472ab38dcb02db5fab96" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44105 } }
I20260925 10:55:44.337261 9931 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:55:44.337384 9931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:55:44.337523 9931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:44.340034 9931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d55d0ea593472ab38dcb02db5fab96" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44105 } }
I20260925 10:55:44.340526 9931 leader_election.cc:302] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46d55d0ea593472ab38dcb02db5fab96; no voters:
I20260925 10:55:44.341104 9931 leader_election.cc:288] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:55:44.341226 9936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:55:44.341866 9936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [term 1 LEADER]: Becoming Leader. State: Replica: 46d55d0ea593472ab38dcb02db5fab96, State: Running, Role: LEADER
I20260925 10:55:44.342396 9936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d55d0ea593472ab38dcb02db5fab96" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44105 } }
I20260925 10:55:44.343037 9931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:55:44.345126 9938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46d55d0ea593472ab38dcb02db5fab96. Latest consensus state: current_term: 1 leader_uuid: "46d55d0ea593472ab38dcb02db5fab96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d55d0ea593472ab38dcb02db5fab96" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44105 } } }
I20260925 10:55:44.345485 9938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:44.345816 9937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46d55d0ea593472ab38dcb02db5fab96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d55d0ea593472ab38dcb02db5fab96" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44105 } } }
I20260925 10:55:44.346128 9937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96 [sys.catalog]: This master's current role is: LEADER
I20260925 10:55:44.348016 9945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:55:44.353590 9945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:55:44.363188 9945 catalog_manager.cc:1347] Generated new cluster ID: e5e24cb0062c4d1883b9f52397e5f05f
I20260925 10:55:44.363343 9945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:55:44.389979 9945 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:55:44.391433 9945 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:55:44.406168 9945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96: Generated new TSK 0
I20260925 10:55:44.407096 9945 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:55:44.477378 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:44105
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:55:44.805660 9957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:44.805953 9957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:44.806046 9957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:44.813246 9957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:44.813371 9957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:44.813491 9957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:55:44.823470 9957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:44105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:44.824801 9957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:44.826133 9957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:44.839715 9962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:44.840309 9965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:44.840374 9963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:44.842276 9957 server_base.cc:1034] running on GCE node
I20260925 10:55:44.842962 9957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:44.844067 9957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:44.845302 9957 hybrid_clock.cc:648] HybridClock initialized: now 1790333744845249 us; error 45 us; skew 500 ppm
I20260925 10:55:44.845629 9957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:44 dist-test-slave-nx7s krb5kdc[9851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333744, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:44.849056 9957 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:55:44.850585 9957 webserver.cc:460] Webserver started at http://127.31.62.129:42279/ using document root <none> and password file <none>
I20260925 10:55:44.851361 9957 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:44.851486 9957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:44.851794 9957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:44.855046 9957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "61072fbe901843e291ca50dcc3b21ef3"
format_stamp: "Formatted at 2026-09-25 10:55:44 on dist-test-slave-nx7s"
server_key: "7d157d089a784ed01cc5d87d3f95f5d7"
server_key_iv: "a871550d0b14b536c1f2b0985ec849b1"
server_key_version: "encryptionkey@0"
I20260925 10:55:44.855865 9957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "61072fbe901843e291ca50dcc3b21ef3"
format_stamp: "Formatted at 2026-09-25 10:55:44 on dist-test-slave-nx7s"
server_key: "7d157d089a784ed01cc5d87d3f95f5d7"
server_key_iv: "a871550d0b14b536c1f2b0985ec849b1"
server_key_version: "encryptionkey@0"
I20260925 10:55:44.861512 9957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 10:55:44.865276 9972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:44.866751 9957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:55:44.866917 9957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "61072fbe901843e291ca50dcc3b21ef3"
format_stamp: "Formatted at 2026-09-25 10:55:44 on dist-test-slave-nx7s"
server_key: "7d157d089a784ed01cc5d87d3f95f5d7"
server_key_iv: "a871550d0b14b536c1f2b0985ec849b1"
server_key_version: "encryptionkey@0"
I20260925 10:55:44.867118 9957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:44.886430 9957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:44.890753 9957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:44.891085 9957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:44.892071 9957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:44.893602 9957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:44.893677 9957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:44.893790 9957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:44.893848 9957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:44.924147 9957 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:45103
I20260925 10:55:44.924243 10084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:45103 every 8 connection(s)
I20260925 10:55:44.925482 9957 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:55:44.929231 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 9957
I20260925 10:55:44.929414 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:55:44.930042 31994 external_mini_cluster.cc:1428] Setting key 573f5722b05264fa36eff25715bfdffd
Sep 25 10:55:44 dist-test-slave-nx7s krb5kdc[9851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333744, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:55:44.947091 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:44.928109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:54721 (local address 127.31.62.190:44105)
0925 10:55:44.928517 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:44.928522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:44.929563 (+ 1041us) server_negotiation.cc:407] Connection header received
0925 10:55:44.931488 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:44.931493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:44.931564 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:44.931685 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:44.933678 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:44.934670 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:44.936473 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:44.937050 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:44.937190 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:44.941226 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:44.941233 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:44.941236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:44.941275 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:44.943836 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:44.944771 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:44.944775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:44.944777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:44.944926 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:44.945379 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:44.945382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:44.945384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:44.945633 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:44.945739 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:44.946579 (+ 840us) server_negotiation.cc:299] Negotiation successful
0925 10:55:44.946795 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":211,"threads_started":1}
I20260925 10:55:44.948520 10088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:44.928516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44105 (local address 127.31.62.129:54721)
0925 10:55:44.929402 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:44.929450 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:44.931228 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:44.932109 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:44.932118 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:44.932838 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:44.933483 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:44.933504 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:44.934858 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:44.934863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:44.936292 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:44.936307 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:44.937296 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:44.937304 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:44.937578 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:44.938302 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:44.938333 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:44.940962 (+ 2629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:44.943985 (+ 3023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:44.944015 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:44.944035 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:44.944631 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:44.945022 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:44.945026 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:44.945030 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:44.945278 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:44.945739 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:44.945748 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:44.946204 (+ 456us) client_negotiation.cc:769] Sending connection context
0925 10:55:44.946489 (+ 285us) client_negotiation.cc:240] Negotiation successful
0925 10:55:44.947067 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":344,"threads_started":1}
I20260925 10:55:44.950318 10085 heartbeater.cc:347] Connected to a master server at 127.31.62.190:44105
I20260925 10:55:44.950695 10085 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:44.951678 10085 heartbeater.cc:511] Master 127.31.62.190:44105 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
I20260925 10:55:44.954195 9896 ts_manager.cc:194] Registered new tserver with Master: 61072fbe901843e291ca50dcc3b21ef3 (127.31.62.129:45103)
I20260925 10:55:44.955740 9896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:54721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:55:44.985642 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:44105
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:55:45.317104 10091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:45.317427 10091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:45.317569 10091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:45.325053 10091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:45.325192 10091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:45.325350 10091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:55:45.335815 10091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:44105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:45.337316 10091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:45.338634 10091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:45.352010 10096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:45.351996 10099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:45.352186 10097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:45.352798 10091 server_base.cc:1034] running on GCE node
I20260925 10:55:45.353577 10091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:45.357193 10091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:45.358481 10091 hybrid_clock.cc:648] HybridClock initialized: now 1790333745358373 us; error 61 us; skew 500 ppm
I20260925 10:55:45.358927 10091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:45 dist-test-slave-nx7s krb5kdc[9851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333745, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:45.363221 10091 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:55:45.374383 10091 webserver.cc:460] Webserver started at http://127.31.62.130:39117/ using document root <none> and password file <none>
I20260925 10:55:45.375408 10091 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:45.375566 10091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:45.375921 10091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:45.379070 10091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "3f44db575b38482fb56df183cdcaa269"
format_stamp: "Formatted at 2026-09-25 10:55:45 on dist-test-slave-nx7s"
server_key: "97206cff375fb2ab2485dc3d6ed6677a"
server_key_iv: "cd508af8b24c877651ada4a1e08e0419"
server_key_version: "encryptionkey@0"
I20260925 10:55:45.379876 10091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "3f44db575b38482fb56df183cdcaa269"
format_stamp: "Formatted at 2026-09-25 10:55:45 on dist-test-slave-nx7s"
server_key: "97206cff375fb2ab2485dc3d6ed6677a"
server_key_iv: "cd508af8b24c877651ada4a1e08e0419"
server_key_version: "encryptionkey@0"
I20260925 10:55:45.385459 10091 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260925 10:55:45.389312 10106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:45.390859 10091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 10:55:45.391010 10091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "3f44db575b38482fb56df183cdcaa269"
format_stamp: "Formatted at 2026-09-25 10:55:45 on dist-test-slave-nx7s"
server_key: "97206cff375fb2ab2485dc3d6ed6677a"
server_key_iv: "cd508af8b24c877651ada4a1e08e0419"
server_key_version: "encryptionkey@0"
I20260925 10:55:45.391193 10091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:45.412508 10091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:45.416884 10091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:45.417148 10091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:45.418218 10091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:45.419485 10091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:45.419559 10091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:45.419649 10091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:45.419691 10091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:45.449922 10091 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:39001
I20260925 10:55:45.449980 10218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:39001 every 8 connection(s)
I20260925 10:55:45.451319 10091 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:55:45.458659 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 10091
I20260925 10:55:45.458858 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:55:45.459291 31994 external_mini_cluster.cc:1428] Setting key bd0a46d51d7598810eaff61744fc4d50
Sep 25 10:55:45 dist-test-slave-nx7s krb5kdc[9851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333745, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:55:45.472254 10221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:45.453930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:49979 (local address 127.31.62.190:44105)
0925 10:55:45.454345 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:45.454349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:45.455482 (+ 1133us) server_negotiation.cc:407] Connection header received
0925 10:55:45.457373 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:45.457378 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:45.457464 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:45.457571 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:45.459329 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:45.460168 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:45.461777 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:45.462193 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:45.462297 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:45.465933 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:45.465938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:45.465941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:45.465975 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:45.468683 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:45.469638 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:45.469643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:45.469645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:45.469809 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:45.470274 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:45.470280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:45.470284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:45.470636 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:45.470784 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:45.471595 (+ 811us) server_negotiation.cc:299] Negotiation successful
0925 10:55:45.471832 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":219,"threads_started":1}
I20260925 10:55:45.473392 10222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:45.454426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44105 (local address 127.31.62.130:49979)
0925 10:55:45.455290 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:45.455345 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:45.457075 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:45.457840 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:45.457849 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:45.458479 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:45.459145 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:45.459165 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:45.460317 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:45.460321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:45.461640 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:45.461653 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:45.462428 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:45.462436 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:45.462668 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:45.463348 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:45.463375 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:45.465674 (+ 2299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:45.468910 (+ 3236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:45.468922 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:45.468943 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:45.469444 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:45.469928 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:45.469934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:45.469938 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:45.470148 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:45.470784 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:45.470793 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:45.471196 (+ 403us) client_negotiation.cc:769] Sending connection context
0925 10:55:45.471490 (+ 294us) client_negotiation.cc:240] Negotiation successful
0925 10:55:45.472092 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":329,"threads_started":1}
I20260925 10:55:45.475121 10219 heartbeater.cc:347] Connected to a master server at 127.31.62.190:44105
I20260925 10:55:45.475442 10219 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:45.476177 10219 heartbeater.cc:511] Master 127.31.62.190:44105 requested a full tablet report, sending...
I20260925 10:55:45.477751 9896 ts_manager.cc:194] Registered new tserver with Master: 3f44db575b38482fb56df183cdcaa269 (127.31.62.130:39001)
I20260925 10:55:45.478878 9896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:49979
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:55:45.508570 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:44105
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:55:45.840631 10225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:55:45.840992 10225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:55:45.841164 10225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:55:45.849117 10225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:55:45.849269 10225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:55:45.849436 10225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:55:45.859668 10225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:44105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:55:45.861204 10225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:55:45.862632 10225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:55:45.876206 10230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:45.876194 10231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:55:45.876274 10233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:55:45.877486 10225 server_base.cc:1034] running on GCE node
I20260925 10:55:45.878203 10225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:55:45.879172 10225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:55:45.880383 10225 hybrid_clock.cc:648] HybridClock initialized: now 1790333745880323 us; error 54 us; skew 500 ppm
I20260925 10:55:45.880705 10225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:55:45 dist-test-slave-nx7s krb5kdc[9851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333745, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:55:45.883958 10225 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:55:45.885753 10225 webserver.cc:460] Webserver started at http://127.31.62.131:33947/ using document root <none> and password file <none>
I20260925 10:55:45.886478 10225 fs_manager.cc:359] Metadata directory not provided
I20260925 10:55:45.886598 10225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:55:45.886891 10225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:55:45.890069 10225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "29facb1dc9614777a05261af17c2aba2"
format_stamp: "Formatted at 2026-09-25 10:55:45 on dist-test-slave-nx7s"
server_key: "f37afd5448065cd2fd2a093e1593c5a4"
server_key_iv: "e3550f177c164a11f88ffc161867d313"
server_key_version: "encryptionkey@0"
I20260925 10:55:45.890851 10225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "29facb1dc9614777a05261af17c2aba2"
format_stamp: "Formatted at 2026-09-25 10:55:45 on dist-test-slave-nx7s"
server_key: "f37afd5448065cd2fd2a093e1593c5a4"
server_key_iv: "e3550f177c164a11f88ffc161867d313"
server_key_version: "encryptionkey@0"
I20260925 10:55:45.896621 10225 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 10:55:45.900329 10240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:45.901947 10225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:55:45.902122 10225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "29facb1dc9614777a05261af17c2aba2"
format_stamp: "Formatted at 2026-09-25 10:55:45 on dist-test-slave-nx7s"
server_key: "f37afd5448065cd2fd2a093e1593c5a4"
server_key_iv: "e3550f177c164a11f88ffc161867d313"
server_key_version: "encryptionkey@0"
I20260925 10:55:45.902330 10225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:55:45.917325 10225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:55:45.921401 10225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:55:45.921674 10225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:55:45.922582 10225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:55:45.924012 10225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:55:45.924115 10225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:45.924228 10225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:55:45.924284 10225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:55:45.955363 10225 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:42265
I20260925 10:55:45.955399 10352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:42265 every 8 connection(s)
I20260925 10:55:45.956632 10225 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:55:45.958922 10085 heartbeater.cc:503] Master 127.31.62.190:44105 was elected leader, sending a full tablet report...
I20260925 10:55:45.961190 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 10225
I20260925 10:55:45.961367 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:55:45.961822 31994 external_mini_cluster.cc:1428] Setting key d950d77e622c76f8d70023143fb9ef8e
Sep 25 10:55:45 dist-test-slave-nx7s krb5kdc[9851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333745, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:55:45.977332 10221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:45.959510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:51099 (local address 127.31.62.190:44105)
0925 10:55:45.959749 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:45.959753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:45.961178 (+ 1425us) server_negotiation.cc:407] Connection header received
0925 10:55:45.963302 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:45.963308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:45.963395 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:45.963505 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:45.965263 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:45.966003 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:45.967628 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:45.968045 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:45.968134 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:45.971547 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:45.971552 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:45.971555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:45.971588 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:45.974047 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:45.974747 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:45.974751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:45.974754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:45.974875 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:45.975424 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:45.975429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:45.975432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:45.975766 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:45.975932 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:45.976781 (+ 849us) server_negotiation.cc:299] Negotiation successful
0925 10:55:45.977027 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260925 10:55:45.978344 10355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:45.960157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44105 (local address 127.31.62.131:51099)
0925 10:55:45.961000 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:45.961052 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:45.963009 (+ 1957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:45.963794 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:45.963803 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:45.964355 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:55:45.965091 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:45.965110 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:45.966137 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:45.966141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:45.967493 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:45.967504 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:45.968225 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:45.968230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:45.968428 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:45.969085 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:55:45.969106 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:55:45.971335 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:55:45.974203 (+ 2868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:45.974210 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:45.974224 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:45.974638 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:45.975011 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:55:45.975015 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:55:45.975018 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:55:45.975297 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:55:45.975888 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:55:45.975895 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:55:45.976381 (+ 486us) client_negotiation.cc:769] Sending connection context
0925 10:55:45.976678 (+ 297us) client_negotiation.cc:240] Negotiation successful
0925 10:55:45.977241 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":334,"threads_started":1}
I20260925 10:55:45.979592 10353 heartbeater.cc:347] Connected to a master server at 127.31.62.190:44105
I20260925 10:55:45.979900 10353 heartbeater.cc:464] Registering TS with master...
I20260925 10:55:45.980640 10353 heartbeater.cc:511] Master 127.31.62.190:44105 requested a full tablet report, sending...
I20260925 10:55:45.982069 9896 ts_manager.cc:194] Registered new tserver with Master: 29facb1dc9614777a05261af17c2aba2 (127.31.62.131:42265)
I20260925 10:55:45.982857 9896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:51099
I20260925 10:55:45.989650 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:55:46.007063 10221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:45.994383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50850 (local address 127.31.62.190:44105)
0925 10:55:45.994549 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:45.994553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:45.995158 (+ 605us) server_negotiation.cc:407] Connection header received
0925 10:55:45.995374 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:45.995380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:45.995465 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:45.995552 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:45.996980 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:45.997695 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:45.998892 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:45.999297 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:45.999385 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:46.001764 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:46.001769 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:46.001772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:46.001807 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:46.004136 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:46.004874 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:46.004877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:46.004880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:46.004990 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:46.005476 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:46.005481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:46.005483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:46.005884 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:46.006146 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:46.006528 (+ 382us) server_negotiation.cc:299] Negotiation successful
0925 10:55:46.006750 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 10:55:46.011139 9896 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:55:46.017979 9896 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:55:46.045384 10365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:46.031196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:39001 (local address 127.0.0.1:35340)
0925 10:55:46.031927 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:46.031976 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:46.032160 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:46.033077 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:46.033085 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:46.033115 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:46.033568 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:46.033590 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.037328 (+ 3738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.037336 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:46.040046 (+ 2710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:46.040067 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.044147 (+ 4080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.044155 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:46.044426 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:46.044494 (+ 68us) client_negotiation.cc:769] Sending connection context
0925 10:55:46.044640 (+ 146us) client_negotiation.cc:240] Negotiation successful
0925 10:55:46.044911 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":289,"threads_started":1}
I20260925 10:55:46.047086 10366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:46.031299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35340 (local address 127.31.62.130:39001)
0925 10:55:46.032392 (+ 1093us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:46.032400 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:46.032431 (+ 31us) server_negotiation.cc:407] Connection header received
0925 10:55:46.032542 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:46.032550 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:46.032812 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:46.033806 (+ 994us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:46.035259 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.037068 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.042103 (+ 5035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.043870 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.046215 (+ 2345us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:46.046447 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:46.046523 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 10:55:46.046700 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":946,"thread_start_us":842,"threads_started":1}
I20260925 10:55:46.050940 10154 tablet_service.cc:1467] Processing CreateTablet for tablet 0df25a20789f4bc998b9537ac5676116 (DEFAULT_TABLE table=test-table [id=abf1a02373b24620a20c979cc30290c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:46.052865 10154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0df25a20789f4bc998b9537ac5676116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:46.053082 10365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:46.032787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:45103 (local address 127.0.0.1:49776)
0925 10:55:46.045678 (+ 12891us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:46.045698 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:46.045811 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:46.047085 (+ 1274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:46.047089 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:46.047113 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:46.047283 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:46.047293 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.049297 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.049302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:46.050617 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:46.050630 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.052267 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.052274 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:46.052435 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:46.052472 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 10:55:46.052537 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 10:55:46.052694 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12793,"mutex_wait_us":25,"thread_start_us":12638,"threads_started":1}
I20260925 10:55:46.053786 10367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:46.032295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:42265 (local address 127.0.0.1:33978)
0925 10:55:46.032755 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:46.032776 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:46.032926 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:46.046375 (+ 13449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:46.046380 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:46.046409 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:46.046802 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:46.046814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.049203 (+ 2389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.049208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:46.051230 (+ 2022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:46.051245 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.053153 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.053160 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:46.053331 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:46.053375 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 10:55:46.053435 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 10:55:46.053537 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":1,"thread_start_us":223,"threads_started":1}
I20260925 10:55:46.053861 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:46.033291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49776 (local address 127.31.62.129:45103)
0925 10:55:46.046243 (+ 12952us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:46.046252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:46.046278 (+ 26us) server_negotiation.cc:407] Connection header received
0925 10:55:46.046603 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:46.046611 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:46.046884 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:46.047469 (+ 585us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:46.047729 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.049179 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.050768 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.052052 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.052984 (+ 932us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:46.053265 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:46.053335 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 10:55:46.053494 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12786,"thread_start_us":13023,"threads_started":1}
I20260925 10:55:46.054678 10369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:46.032754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33978 (local address 127.31.62.131:42265)
0925 10:55:46.045646 (+ 12892us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:46.045657 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:46.045692 (+ 35us) server_negotiation.cc:407] Connection header received
0925 10:55:46.045797 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:46.045805 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:46.046106 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:46.047040 (+ 934us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:46.047308 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.049064 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.051395 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.052911 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.054094 (+ 1183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:46.054270 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:46.054326 (+ 56us) server_negotiation.cc:299] Negotiation successful
0925 10:55:46.054438 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12681,"thread_start_us":12796,"threads_started":1}
I20260925 10:55:46.057412 10020 tablet_service.cc:1467] Processing CreateTablet for tablet 0df25a20789f4bc998b9537ac5676116 (DEFAULT_TABLE table=test-table [id=abf1a02373b24620a20c979cc30290c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:46.058810 10020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0df25a20789f4bc998b9537ac5676116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:46.058341 10288 tablet_service.cc:1467] Processing CreateTablet for tablet 0df25a20789f4bc998b9537ac5676116 (DEFAULT_TABLE table=test-table [id=abf1a02373b24620a20c979cc30290c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:55:46.059720 10288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0df25a20789f4bc998b9537ac5676116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:55:46.068701 10371 tablet_bootstrap.cc:492] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269: Bootstrap starting.
I20260925 10:55:46.070685 10372 tablet_bootstrap.cc:492] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3: Bootstrap starting.
I20260925 10:55:46.072947 10371 tablet_bootstrap.cc:654] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:46.074488 10373 tablet_bootstrap.cc:492] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2: Bootstrap starting.
I20260925 10:55:46.074533 10371 log.cc:824] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:46.074239 10372 tablet_bootstrap.cc:654] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:46.076032 10372 log.cc:824] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:46.077703 10373 tablet_bootstrap.cc:654] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2: Neither blocks nor log segments found. Creating new log.
I20260925 10:55:46.077966 10371 tablet_bootstrap.cc:492] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269: No bootstrap required, opened a new log
I20260925 10:55:46.078253 10371 ts_tablet_manager.cc:1397] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260925 10:55:46.078641 10372 tablet_bootstrap.cc:492] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3: No bootstrap required, opened a new log
I20260925 10:55:46.078872 10372 ts_tablet_manager.cc:1397] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260925 10:55:46.078853 10373 log.cc:824] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2: Log is configured to *not* fsync() on all Append() calls
I20260925 10:55:46.081839 10373 tablet_bootstrap.cc:492] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2: No bootstrap required, opened a new log
I20260925 10:55:46.082098 10373 ts_tablet_manager.cc:1397] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260925 10:55:46.086588 10372 raft_consensus.cc:348] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f44db575b38482fb56df183cdcaa269" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39001 } } peers { permanent_uuid: "29facb1dc9614777a05261af17c2aba2" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42265 } } peers { permanent_uuid: "61072fbe901843e291ca50dcc3b21ef3" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45103 } }
I20260925 10:55:46.086997 10372 raft_consensus.cc:374] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:46.087111 10372 raft_consensus.cc:729] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61072fbe901843e291ca50dcc3b21ef3, State: Initialized, Role: FOLLOWER
I20260925 10:55:46.087709 10372 consensus_queue.cc:260] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f44db575b38482fb56df183cdcaa269" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39001 } } peers { permanent_uuid: "29facb1dc9614777a05261af17c2aba2" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42265 } } peers { permanent_uuid: "61072fbe901843e291ca50dcc3b21ef3" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45103 } }
I20260925 10:55:46.088100 10371 raft_consensus.cc:348] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f44db575b38482fb56df183cdcaa269" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39001 } } peers { permanent_uuid: "29facb1dc9614777a05261af17c2aba2" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42265 } } peers { permanent_uuid: "61072fbe901843e291ca50dcc3b21ef3" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45103 } }
I20260925 10:55:46.089319 10372 ts_tablet_manager.cc:1428] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3: Time spent starting tablet: real 0.010s user 0.006s sys 0.006s
I20260925 10:55:46.089463 10371 raft_consensus.cc:374] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:46.089633 10371 raft_consensus.cc:729] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f44db575b38482fb56df183cdcaa269, State: Initialized, Role: FOLLOWER
I20260925 10:55:46.090224 10371 consensus_queue.cc:260] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f44db575b38482fb56df183cdcaa269" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39001 } } peers { permanent_uuid: "29facb1dc9614777a05261af17c2aba2" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42265 } } peers { permanent_uuid: "61072fbe901843e291ca50dcc3b21ef3" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45103 } }
I20260925 10:55:46.091339 10219 heartbeater.cc:503] Master 127.31.62.190:44105 was elected leader, sending a full tablet report...
I20260925 10:55:46.092095 10371 ts_tablet_manager.cc:1428] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260925 10:55:46.092077 10373 raft_consensus.cc:348] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f44db575b38482fb56df183cdcaa269" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39001 } } peers { permanent_uuid: "29facb1dc9614777a05261af17c2aba2" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42265 } } peers { permanent_uuid: "61072fbe901843e291ca50dcc3b21ef3" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45103 } }
I20260925 10:55:46.092592 10373 raft_consensus.cc:374] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:55:46.092700 10373 raft_consensus.cc:729] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29facb1dc9614777a05261af17c2aba2, State: Initialized, Role: FOLLOWER
I20260925 10:55:46.093343 10373 consensus_queue.cc:260] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f44db575b38482fb56df183cdcaa269" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39001 } } peers { permanent_uuid: "29facb1dc9614777a05261af17c2aba2" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42265 } } peers { permanent_uuid: "61072fbe901843e291ca50dcc3b21ef3" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45103 } }
I20260925 10:55:46.094609 10353 heartbeater.cc:503] Master 127.31.62.190:44105 was elected leader, sending a full tablet report...
I20260925 10:55:46.095335 10373 ts_tablet_manager.cc:1428] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
W20260925 10:55:46.178004 10086 tablet.cc:2369] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:55:46.186863 10378 raft_consensus.cc:482] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:55:46.187144 10378 raft_consensus.cc:504] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f44db575b38482fb56df183cdcaa269" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39001 } } peers { permanent_uuid: "29facb1dc9614777a05261af17c2aba2" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42265 } } peers { permanent_uuid: "61072fbe901843e291ca50dcc3b21ef3" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45103 } }
I20260925 10:55:46.189028 10378 leader_election.cc:288] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29facb1dc9614777a05261af17c2aba2 (127.31.62.131:42265), 61072fbe901843e291ca50dcc3b21ef3 (127.31.62.129:45103)
I20260925 10:55:46.196242 10369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:46.189304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:35035 (local address 127.31.62.131:42265)
0925 10:55:46.189541 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:46.189547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:46.189934 (+ 387us) server_negotiation.cc:407] Connection header received
0925 10:55:46.190114 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:46.190119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:46.190203 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:46.190289 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:46.191051 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.192130 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.194369 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.195317 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.195469 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:46.195544 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:46.195863 (+ 319us) server_negotiation.cc:299] Negotiation successful
0925 10:55:46.195988 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 10:55:46.196357 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:46.190066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:58865 (local address 127.31.62.129:45103)
0925 10:55:46.190254 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:46.190259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:46.190599 (+ 340us) server_negotiation.cc:407] Connection header received
0925 10:55:46.190711 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:46.190716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:46.190789 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:46.190878 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:55:46.191657 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.192494 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.194555 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.195460 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.195621 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:46.195698 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:46.195999 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 10:55:46.196125 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 10:55:46.196398 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:46.189234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:42265 (local address 127.31.62.130:35035)
0925 10:55:46.189831 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:46.189847 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:46.189981 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:46.190346 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:46.190352 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:46.190383 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:46.190768 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:46.190781 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.192255 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.192261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:46.194149 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:46.194164 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.195519 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.195525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:46.195694 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:46.195726 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 10:55:46.195812 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 10:55:46.195978 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":52,"thread_start_us":273,"threads_started":1}
I20260925 10:55:46.196606 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:46.189933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:45103 (local address 127.31.62.130:58865)
0925 10:55:46.190482 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0925 10:55:46.190498 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:55:46.190611 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:55:46.190972 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:55:46.190978 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:55:46.191007 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:55:46.191425 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:46.191438 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.192657 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.192661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:46.194353 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:55:46.194366 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.195690 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.195696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:55:46.195845 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:46.195874 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 10:55:46.195975 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 10:55:46.196139 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":257,"threads_started":1}
I20260925 10:55:46.197533 10040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0df25a20789f4bc998b9537ac5676116" candidate_uuid: "3f44db575b38482fb56df183cdcaa269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61072fbe901843e291ca50dcc3b21ef3" is_pre_election: true
I20260925 10:55:46.197530 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0df25a20789f4bc998b9537ac5676116" candidate_uuid: "3f44db575b38482fb56df183cdcaa269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29facb1dc9614777a05261af17c2aba2" is_pre_election: true
I20260925 10:55:46.198112 10308 raft_consensus.cc:2454] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f44db575b38482fb56df183cdcaa269 in term 0.
I20260925 10:55:46.198112 10040 raft_consensus.cc:2454] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f44db575b38482fb56df183cdcaa269 in term 0.
I20260925 10:55:46.198776 10107 leader_election.cc:302] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29facb1dc9614777a05261af17c2aba2, 3f44db575b38482fb56df183cdcaa269; no voters:
I20260925 10:55:46.199119 10378 raft_consensus.cc:2781] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:55:46.199262 10378 raft_consensus.cc:482] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:55:46.199368 10378 raft_consensus.cc:3037] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:46.201537 10378 raft_consensus.cc:504] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f44db575b38482fb56df183cdcaa269" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39001 } } peers { permanent_uuid: "29facb1dc9614777a05261af17c2aba2" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42265 } } peers { permanent_uuid: "61072fbe901843e291ca50dcc3b21ef3" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45103 } }
I20260925 10:55:46.202116 10378 leader_election.cc:288] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [CANDIDATE]: Term 1 election: Requested vote from peers 29facb1dc9614777a05261af17c2aba2 (127.31.62.131:42265), 61072fbe901843e291ca50dcc3b21ef3 (127.31.62.129:45103)
I20260925 10:55:46.202595 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0df25a20789f4bc998b9537ac5676116" candidate_uuid: "3f44db575b38482fb56df183cdcaa269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29facb1dc9614777a05261af17c2aba2"
I20260925 10:55:46.202625 10040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0df25a20789f4bc998b9537ac5676116" candidate_uuid: "3f44db575b38482fb56df183cdcaa269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61072fbe901843e291ca50dcc3b21ef3"
I20260925 10:55:46.202813 10040 raft_consensus.cc:3037] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:55:46.202813 10308 raft_consensus.cc:3037] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2 [term 0 FOLLOWER]: Advancing to term 1
W20260925 10:55:46.203205 10220 tablet.cc:2369] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:55:46.206003 10308 raft_consensus.cc:2454] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f44db575b38482fb56df183cdcaa269 in term 1.
I20260925 10:55:46.206003 10040 raft_consensus.cc:2454] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f44db575b38482fb56df183cdcaa269 in term 1.
I20260925 10:55:46.206437 10107 leader_election.cc:302] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29facb1dc9614777a05261af17c2aba2, 3f44db575b38482fb56df183cdcaa269; no voters:
I20260925 10:55:46.206694 10378 raft_consensus.cc:2781] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:55:46.207139 10378 raft_consensus.cc:686] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [term 1 LEADER]: Becoming Leader. State: Replica: 3f44db575b38482fb56df183cdcaa269, State: Running, Role: LEADER
I20260925 10:55:46.207656 10378 consensus_queue.cc:237] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f44db575b38482fb56df183cdcaa269" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39001 } } peers { permanent_uuid: "29facb1dc9614777a05261af17c2aba2" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42265 } } peers { permanent_uuid: "61072fbe901843e291ca50dcc3b21ef3" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45103 } }
W20260925 10:55:46.208195 10354 tablet.cc:2369] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:55:46.212062 9895 catalog_manager.cc:5495] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f44db575b38482fb56df183cdcaa269 (127.31.62.130). New cstate: current_term: 1 leader_uuid: "3f44db575b38482fb56df183cdcaa269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f44db575b38482fb56df183cdcaa269" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29facb1dc9614777a05261af17c2aba2" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61072fbe901843e291ca50dcc3b21ef3" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45103 } health_report { overall_health: UNKNOWN } } }
I20260925 10:55:46.238749 10366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:46.233510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35350 (local address 127.31.62.130:39001)
0925 10:55:46.233736 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:46.233742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:46.233765 (+ 23us) server_negotiation.cc:407] Connection header received
0925 10:55:46.233958 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:46.233961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:46.234073 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:46.234214 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:55:46.234614 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.235373 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.236647 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.237100 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.237210 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:46.237665 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:55:46.238176 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:55:46.238310 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:46.238408 (+ 98us) server_negotiation.cc:299] Negotiation successful
0925 10:55:46.238488 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 10:55:46.247609 10308 raft_consensus.cc:1261] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2 [term 1 FOLLOWER]: Refusing update from remote peer 3f44db575b38482fb56df183cdcaa269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:55:46.247609 10040 raft_consensus.cc:1261] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3 [term 1 FOLLOWER]: Refusing update from remote peer 3f44db575b38482fb56df183cdcaa269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:55:46.248698 10382 consensus_queue.cc:1035] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29facb1dc9614777a05261af17c2aba2" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:46.249123 10378 consensus_queue.cc:1035] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61072fbe901843e291ca50dcc3b21ef3" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 45103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:55:46.260576 10385 mvcc.cc:205] Tried to move back new op lower bound from 7333207024619098112 to 7333207024470704128. Current Snapshot: MvccSnapshot[applied={T|T < 7333207024619098112}]
I20260925 10:55:46.264971 10387 mvcc.cc:205] Tried to move back new op lower bound from 7333207024619098112 to 7333207024470704128. Current Snapshot: MvccSnapshot[applied={T|T < 7333207024619098112}]
I20260925 10:55:46.273895 9895 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:55:46.274153 9895 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:55:46.277904 9895 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46d55d0ea593472ab38dcb02db5fab96: Sending DeleteTablet for 3 replicas of tablet 0df25a20789f4bc998b9537ac5676116
I20260925 10:55:46.279127 10154 tablet_service.cc:1514] Processing DeleteTablet for tablet 0df25a20789f4bc998b9537ac5676116 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:46 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:35340
I20260925 10:55:46.279315 10288 tablet_service.cc:1514] Processing DeleteTablet for tablet 0df25a20789f4bc998b9537ac5676116 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:46 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:33978
I20260925 10:55:46.279857 10020 tablet_service.cc:1514] Processing DeleteTablet for tablet 0df25a20789f4bc998b9537ac5676116 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:55:46 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:49776
I20260925 10:55:46.280071 10394 tablet_replica.cc:330] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269: stopping tablet replica
I20260925 10:55:46.280535 10394 raft_consensus.cc:2229] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:55:46.282284 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 9957
I20260925 10:55:46.290079 10396 tablet_replica.cc:330] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3: stopping tablet replica
I20260925 10:55:46.290849 10396 raft_consensus.cc:2229] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:46.291596 10396 raft_consensus.cc:2258] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:46.292318 10394 raft_consensus.cc:2258] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:46.294183 10396 ts_tablet_manager.cc:1905] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:46.296993 10394 ts_tablet_manager.cc:1905] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:46.301362 10395 tablet_replica.cc:330] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2: stopping tablet replica
I20260925 10:55:46.301914 10395 raft_consensus.cc:2229] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:55:46.302546 10395 raft_consensus.cc:2258] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:55:46.303613 10396 ts_tablet_manager.cc:1918] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:55:46.305794 10395 ts_tablet_manager.cc:1905] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:55:46.312640 10395 ts_tablet_manager.cc:1918] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:55:46.312822 10394 ts_tablet_manager.cc:1918] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:55:46.312990 10396 log.cc:1196] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/0df25a20789f4bc998b9537ac5676116
I20260925 10:55:46.313601 10396 ts_tablet_manager.cc:1939] T 0df25a20789f4bc998b9537ac5676116 P 61072fbe901843e291ca50dcc3b21ef3: Deleting consensus metadata
I20260925 10:55:46.315734 10395 log.cc:1196] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/0df25a20789f4bc998b9537ac5676116
I20260925 10:55:46.315673 9880 catalog_manager.cc:4841] TS 61072fbe901843e291ca50dcc3b21ef3 (127.31.62.129:45103): tablet 0df25a20789f4bc998b9537ac5676116 (table test-table [id=abf1a02373b24620a20c979cc30290c2]) successfully deleted
I20260925 10:55:46.316216 10394 log.cc:1196] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/0df25a20789f4bc998b9537ac5676116
Sep 25 10:55:46 dist-test-slave-nx7s krb5kdc[9851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:55:46.316794 10394 ts_tablet_manager.cc:1939] T 0df25a20789f4bc998b9537ac5676116 P 3f44db575b38482fb56df183cdcaa269: Deleting consensus metadata
I20260925 10:55:46.318276 10395 ts_tablet_manager.cc:1939] T 0df25a20789f4bc998b9537ac5676116 P 29facb1dc9614777a05261af17c2aba2: Deleting consensus metadata
I20260925 10:55:46.320113 9880 catalog_manager.cc:4841] TS 29facb1dc9614777a05261af17c2aba2 (127.31.62.131:42265): tablet 0df25a20789f4bc998b9537ac5676116 (table test-table [id=abf1a02373b24620a20c979cc30290c2]) successfully deleted
I20260925 10:55:46.321390 9880 catalog_manager.cc:4841] TS 3f44db575b38482fb56df183cdcaa269 (127.31.62.130:39001): tablet 0df25a20789f4bc998b9537ac5676116 (table test-table [id=abf1a02373b24620a20c979cc30290c2]) successfully deleted
I20260925 10:55:46.324517 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:46.284065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49786 (local address 127.31.62.129:45103)
0925 10:55:46.284261 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:46.284269 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:46.298125 (+ 13856us) server_negotiation.cc:407] Connection header received
0925 10:55:46.302017 (+ 3892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:46.302024 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:46.302117 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:46.302193 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:46.303961 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.305048 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.308066 (+ 3018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.308682 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.308795 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:46.317992 (+ 9197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:46.318008 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:46.318026 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:46.318074 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:46.321643 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:46.322351 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:46.322359 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:46.322361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:46.322489 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:46.322868 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:46.322872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:46.322874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:46.323270 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:46.323529 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:46.323860 (+ 331us) server_negotiation.cc:299] Negotiation successful
0925 10:55:46.324070 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 10:55:46.325219 10080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:46.414937 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 9957
I20260925 10:55:46.429679 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 10091
Sep 25 10:55:46 dist-test-slave-nx7s krb5kdc[9851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:55:46.447072 10366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:46.430562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35358 (local address 127.31.62.130:39001)
0925 10:55:46.430714 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:46.430718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:46.430739 (+ 21us) server_negotiation.cc:407] Connection header received
0925 10:55:46.430852 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:46.430855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:46.430953 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:46.431034 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:46.432625 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.433483 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.435378 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.435847 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.435949 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:46.440975 (+ 5026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:46.440992 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:46.441008 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:46.441063 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:46.444108 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:46.444916 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:46.444924 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:46.444927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:46.445093 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:46.445504 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:46.445508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:46.445511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:46.445864 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:46.446199 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:46.446489 (+ 290us) server_negotiation.cc:299] Negotiation successful
0925 10:55:46.446702 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 10:55:46.447798 10214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:46.560143 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 10091
I20260925 10:55:46.575098 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 10225
Sep 25 10:55:46 dist-test-slave-nx7s krb5kdc[9851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:55:46.590268 10369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:55:46.575923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33982 (local address 127.31.62.131:42265)
0925 10:55:46.576100 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 10:55:46.576106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:55:46.576125 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:55:46.576295 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:55:46.576299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:55:46.576361 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:55:46.576443 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:55:46.577809 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.578525 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.579789 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:55:46.580177 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:55:46.580298 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:55:46.584581 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:55:46.584598 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:55:46.584614 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:55:46.584658 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:55:46.587443 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:46.588170 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:46.588178 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:46.588180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:46.588357 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:55:46.588723 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:55:46.588728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:55:46.588731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:55:46.589126 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:55:46.589407 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0925 10:55:46.589731 (+ 324us) server_negotiation.cc:299] Negotiation successful
0925 10:55:46.589943 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 10:55:46.590909 10348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:46.679085 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 10225
I20260925 10:55:46.691900 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 9864
I20260925 10:55:46.693030 9926 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:55:46.798612 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 9864
2026-09-25T10:55:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4176 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:55:46 dist-test-slave-nx7s krb5kdc[10404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:55:46 dist-test-slave-nx7s krb5kdc[10404](info): set up 2 sockets
Sep 25 10:55:46 dist-test-slave-nx7s krb5kdc[10404](info): commencing operation
krb5kdc: starting...
W20260925 10:55:47.821971 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:55:47 dist-test-slave-nx7s krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:55:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:55:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/rangeradmin_127.31.62.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/rangeradmin_127.31.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/rangerlookup_127.31.62.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/rangerlookup_127.31.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/HTTP_127.31.62.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/HTTP_127.31.62.148@KRBTEST.COM.keytab.
I20260925 10:55:48.144307 31994 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task9q_Uwc/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/postgres -l logfile start
2026-09-25 10:55:50.759 UTC [10434] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 10:55:50.759 UTC [10434] LOG: listening on IPv4 address "127.31.62.148", port 60849
2026-09-25 10:55:50.763 UTC [10434] LOG: listening on Unix socket "/tmp/.s.PGSQL.60849"
2026-09-25 10:55:50.786 UTC [10437] LOG: database system was shut down at 2026-09-25 10:55:49 UTC
2026-09-25 10:55:50.789 UTC [10434] LOG: database system is ready to accept connections
I20260925 10:55:51.689649 31994 mini_postgres.cc:96] Postgres bound to 60849
2026-09-25 10:55:51.706 UTC [10445] FATAL: database "slave" does not exist
127.31.62.148:60849 - accepting connections
I20260925 10:55:51.706984 31994 mini_ranger.cc:161] Starting Ranger...
I20260925 10:55:51.733176 31994 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 10:55:52.244282 31994 mini_ranger.cc:87] Created ranger Postgres database
I20260925 10:55:52.244441 31994 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-admin
2026-09-25 10:55:52,538 [I] DB FLAVOR :POSTGRES
2026-09-25 10:55:52,538 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 10:55:52,538 [I] Checking connection..
2026-09-25 10:55:52,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 10:55:52,796 [I] Checking connection passed.
2026-09-25 10:55:52,796 [I] --------- Verifying version history table ---------
2026-09-25 10:55:52,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 10:55:53,034 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 10:55:53,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 10:55:53,299 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 10:55:53,299 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 10:55:53,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 10:55:53.560 UTC [10509] WARNING: there is no transaction in progress
2026-09-25 10:55:53,566 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 10:55:53,567 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 10:55:53,800 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 10:55:53,800 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 10:55:53,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 10:55:54,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 10:55:54,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-nx7s.c.gcp-upstream.internal','N') ;"
2026-09-25 10:55:54,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 10:55:54,825 [I] Table x_portal_user does not exist in database ranger
2026-09-25 10:55:54,825 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 10:55:55,083 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 10:55:55,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 10:55:55,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 10:55:55,586 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 10:55:55,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 10:55:56.626 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:56.644 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:56.661 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:56.678 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:56.696 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:56.716 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:56.778 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:56.787 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:56.803 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:56.814 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:56.825 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:56.841 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:56.852 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:56.859 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:56.869 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:56.876 UTC [10617] WARNING: there is no transaction in progress
2026-09-25 10:55:57,644 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 10:55:57,644 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 10:55:57,897 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 10:55:57,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 10:55:58,146 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 10:55:58,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 10:55:58,391 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 10:55:58,391 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 10:55:58,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 10:55:58,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-nx7s.c.gcp-upstream.internal';"
2026-09-25 10:55:59,129 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 10:55:59,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 10:55:59,367 [I] DB_PATCHES have already been applied
I20260925 10:55:59.373807 31994 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-admin:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task9q_Uwc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 10:55:59.373935 31994 mini_ranger.cc:193] Using host: 127.31.62.148
I20260925 10:55:59.388808 31994 mini_ranger.cc:227] Ranger admin URL: http://127.31.62.148:60467
Sep 25, 2026 10:55:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 10:55:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 10:55:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 10:55:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 10:55:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 10:56:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 10:56:00 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 10:56:00.386991 31994 mini_ranger.cc:160] Time spent starting Ranger: real 8.680s user 0.006s sys 0.037s
Sep 25, 2026 10:56:21 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 10:56:21 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 10:56:21 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 10:56:21 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 10:56:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 10:56:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 10:56:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 10:56:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 10:56:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 10:56:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 10:56:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 10:56:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 10:56:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 10:56:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 10:56:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 10:56:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 10:56:21 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 10:56:22 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 10:56:22.888712 31994 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/rangerkms_127.31.62.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/rangerkms_127.31.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.62.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.62.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.62.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/HTTP_127.31.62.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.62.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/HTTP_127.31.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 10:56:23 dist-test-slave-nx7s krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333783, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 10:56:23.048924 31994 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 10:56:23.068885 31994 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 10:56:23.958097 31994 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 10:56:23.958218 31994 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-kms
I20260925 10:56:23.958365 31994 mini_ranger_kms.cc:227] Using postgres at 127.31.62.148:60849
2026-09-25 10:56:24,359 [I] DB FLAVOR :POSTGRES
2026-09-25 10:56:24,359 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 10:56:24,359 [I] Checking connection
2026-09-25 10:56:24,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 10:56:24,610 [I] connection success
2026-09-25 10:56:24,611 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 10:56:24,611 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 10:56:24,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 10:56:24,842 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 10:56:24,842 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 10:56:24,842 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 10:56:24,842 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:60849/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 10:56:25,099 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 10:56:25.586388 31994 mini_ranger_kms.cc:310] Created kms service
I20260925 10:56:25.758487 31994 mini_ranger_kms.cc:326] Created kudu user
I20260925 10:56:25.807363 31994 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 10:56:25 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333783, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:56:26.207942 31994 mini_ranger_kms.cc:384] Added ranger policy
I20260925 10:56:26.208271 31994 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-kms:/tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 10:56:26.208338 31994 mini_ranger_kms.cc:240] Using host: 127.31.62.148
I20260925 10:56:26.226378 31994 mini_ranger_kms.cc:276] Ranger KMS PID: 10798
I20260925 10:56:26.226583 31994 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.62.148:35681
Sep 25, 2026 10:56:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 10:56:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 10:56:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 10:56:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 10:56:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 10:56:27 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 10:56:28.354058 31994 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.305s user 0.110s sys 0.319s
I20260925 10:56:28.354305 31994 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 10:56:28.354502 31994 mini_ranger_kms.cc:401] 127.31.62.148:35681/kms/v1/keys
Sep 25 10:56:28 dist-test-slave-nx7s krb5kdc[10404](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790333788, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.62.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 10:56:28 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333788, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.62.148@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
Sep 25 10:56:29 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333788, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.62.148@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
Sep 25 10:56:31 dist-test-slave-nx7s krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:56:31.777868 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41231
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:41231
--ranger_config_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.62.148:35681/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:56:32.172674 10848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:56:32.173059 10848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:56:32.173192 10848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:56:32.181145 10848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:56:32.181286 10848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:56:32.181362 10848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:56:32.181437 10848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:56:32.181500 10848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:56:32.192057 10848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.62.148:35681/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:41231
--ranger_config_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:56:32.193455 10848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:56:32.194932 10848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:56:32.207072 10856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:56:32.207234 10854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:56:32.207233 10853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:56:32.208369 10848 server_base.cc:1034] running on GCE node
I20260925 10:56:32.209235 10848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:56:32.210762 10848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:56:32.211964 10848 hybrid_clock.cc:648] HybridClock initialized: now 1790333792211907 us; error 57 us; skew 500 ppm
I20260925 10:56:32.212330 10848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:56:32 dist-test-slave-nx7s krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:56:32.219894 10848 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:56:32.221784 10848 webserver.cc:460] Webserver started at http://127.31.62.190:42067/ using document root <none> and password file <none>
I20260925 10:56:32.222553 10848 fs_manager.cc:359] Metadata directory not provided
I20260925 10:56:32.222698 10848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:56:32.223054 10848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 10:56:32 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333792, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:56:32.282150 10848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "77787ba4951b447eb09298a0422e7aa2"
format_stamp: "Formatted at 2026-09-25 10:56:32 on dist-test-slave-nx7s"
server_key: "b567848f65c8bcace2d2a3e221badac8"
server_key_iv: "ae9b0177aaa239343134e11533bf0fda"
server_key_version: "kuduclusterkey@0"
I20260925 10:56:32.283012 10848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "77787ba4951b447eb09298a0422e7aa2"
format_stamp: "Formatted at 2026-09-25 10:56:32 on dist-test-slave-nx7s"
server_key: "b567848f65c8bcace2d2a3e221badac8"
server_key_iv: "ae9b0177aaa239343134e11533bf0fda"
server_key_version: "kuduclusterkey@0"
I20260925 10:56:32.288897 10848 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260925 10:56:32.309697 10863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:56:32.311211 10848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:56:32.311383 10848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "77787ba4951b447eb09298a0422e7aa2"
format_stamp: "Formatted at 2026-09-25 10:56:32 on dist-test-slave-nx7s"
server_key: "b567848f65c8bcace2d2a3e221badac8"
server_key_iv: "ae9b0177aaa239343134e11533bf0fda"
server_key_version: "kuduclusterkey@0"
I20260925 10:56:32.311594 10848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:56:32.326303 10848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:56:32.331152 10848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:56:32.331449 10848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:56:32.351797 10848 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:41231
I20260925 10:56:32.351797 10914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:41231 every 8 connection(s)
I20260925 10:56:32.353404 10848 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:56:32.358449 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:56:32.358918 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 10848
I20260925 10:56:32.359107 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:56:32.359754 31994 external_mini_cluster.cc:1428] Setting key 9f4daea54fe29686c8f889c80b90f0e2
I20260925 10:56:32.372985 10915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2: Bootstrap starting.
Sep 25 10:56:32 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:56:32.377502 10915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2: Neither blocks nor log segments found. Creating new log.
I20260925 10:56:32.378938 10915 log.cc:824] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2: Log is configured to *not* fsync() on all Append() calls
I20260925 10:56:32.382632 10915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2: No bootstrap required, opened a new log
I20260925 10:56:32.385638 10918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:32.361992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57838 (local address 127.31.62.190:41231)
0925 10:56:32.362934 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0925 10:56:32.362947 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:56:32.363003 (+ 56us) server_negotiation.cc:407] Connection header received
0925 10:56:32.364800 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:56:32.364833 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:56:32.365404 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:56:32.365793 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:56:32.367472 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:32.369053 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:32.370914 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:32.371759 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:32.371909 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:32.376709 (+ 4800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:56:32.376719 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:56:32.376738 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:56:32.376785 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:56:32.380395 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:32.381364 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:32.381373 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:32.381383 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:32.381604 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:32.382102 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:32.382107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:32.382110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:32.382853 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:56:32.383146 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0925 10:56:32.383692 (+ 546us) server_negotiation.cc:299] Negotiation successful
0925 10:56:32.384127 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":393,"threads_started":1}
I20260925 10:56:32.391709 10915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77787ba4951b447eb09298a0422e7aa2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41231 } }
I20260925 10:56:32.392095 10915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:56:32.392172 10915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77787ba4951b447eb09298a0422e7aa2, State: Initialized, Role: FOLLOWER
I20260925 10:56:32.392709 10915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77787ba4951b447eb09298a0422e7aa2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41231 } }
I20260925 10:56:32.392967 10915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:56:32.393093 10915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:56:32.393258 10915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:56:32.395560 10915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77787ba4951b447eb09298a0422e7aa2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41231 } }
I20260925 10:56:32.396104 10915 leader_election.cc:302] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77787ba4951b447eb09298a0422e7aa2; no voters:
I20260925 10:56:32.396700 10915 leader_election.cc:288] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:56:32.396886 10920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:56:32.397603 10920 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [term 1 LEADER]: Becoming Leader. State: Replica: 77787ba4951b447eb09298a0422e7aa2, State: Running, Role: LEADER
I20260925 10:56:32.398175 10920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77787ba4951b447eb09298a0422e7aa2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41231 } }
I20260925 10:56:32.398767 10915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:56:32.400743 10922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77787ba4951b447eb09298a0422e7aa2. Latest consensus state: current_term: 1 leader_uuid: "77787ba4951b447eb09298a0422e7aa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77787ba4951b447eb09298a0422e7aa2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41231 } } }
I20260925 10:56:32.401453 10922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [sys.catalog]: This master's current role is: LEADER
I20260925 10:56:32.401763 10921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77787ba4951b447eb09298a0422e7aa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77787ba4951b447eb09298a0422e7aa2" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41231 } } }
I20260925 10:56:32.402629 10921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2 [sys.catalog]: This master's current role is: LEADER
I20260925 10:56:32.403399 10926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:56:32.409652 10926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:56:32.410300 10915 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 10:56:32.432512 10926 catalog_manager.cc:1347] Generated new cluster ID: 434ead2b93814deface7f1a9fa831631
I20260925 10:56:32.432685 10926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:56:32.457664 10926 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:56:32.459807 10926 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:56:32.475948 10926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2: Generated new TSK 0
I20260925 10:56:32.477180 10926 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:56:33.301256 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.62.148:35681/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41231
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 10:56:33 dist-test-slave-nx7s krb5kdc[10404](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790333793, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 10:56:33.758006 10960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:56:33.758546 10960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:56:33.758765 10960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:56:33.767130 10960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:56:33.767273 10960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:56:33.767433 10960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:56:33.779040 10960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.62.148:35681/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41231
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:56:33.780448 10960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:56:33.782081 10960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:56:33.800573 10970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:56:33.801020 10971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:56:33.806375 10960 server_base.cc:1034] running on GCE node
W20260925 10:56:33.806792 10973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:56:33.807569 10960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:56:33.809291 10960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:56:33.810581 10960 hybrid_clock.cc:648] HybridClock initialized: now 1790333793810487 us; error 96 us; skew 500 ppm
I20260925 10:56:33.810935 10960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:56:33 dist-test-slave-nx7s krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333793, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:56:33.814288 10960 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:56:33.818305 10960 webserver.cc:460] Webserver started at http://127.31.62.129:35277/ using document root <none> and password file <none>
I20260925 10:56:33.819146 10960 fs_manager.cc:359] Metadata directory not provided
I20260925 10:56:33.819403 10960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:56:33.819840 10960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 10:56:33 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333793, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:56:33.838941 10960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "442bc0f447c146cfa3324e36457ec97f"
format_stamp: "Formatted at 2026-09-25 10:56:33 on dist-test-slave-nx7s"
server_key: "581a875c0f37aa21b8fc8bd24c6a8258"
server_key_iv: "fb1177447c4d49df4ced1860e4154a47"
server_key_version: "kuduclusterkey@0"
I20260925 10:56:33.840327 10960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "442bc0f447c146cfa3324e36457ec97f"
format_stamp: "Formatted at 2026-09-25 10:56:33 on dist-test-slave-nx7s"
server_key: "581a875c0f37aa21b8fc8bd24c6a8258"
server_key_iv: "fb1177447c4d49df4ced1860e4154a47"
server_key_version: "kuduclusterkey@0"
I20260925 10:56:33.852082 10960 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.006s sys 0.003s
I20260925 10:56:33.891739 10980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:56:33.893997 10960 fs_manager.cc:725] Time spent opening block manager: real 0.021s user 0.005s sys 0.016s
I20260925 10:56:33.894230 10960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "442bc0f447c146cfa3324e36457ec97f"
format_stamp: "Formatted at 2026-09-25 10:56:33 on dist-test-slave-nx7s"
server_key: "581a875c0f37aa21b8fc8bd24c6a8258"
server_key_iv: "fb1177447c4d49df4ced1860e4154a47"
server_key_version: "kuduclusterkey@0"
I20260925 10:56:33.894511 10960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:56:33.939450 10960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:56:33.945204 10960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:56:33.946161 10960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:56:33.953660 10960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:56:33.975245 10960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:56:33.975458 10960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260925 10:56:33.975697 10960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:56:33.975873 10960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 10:56:34 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333793, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:56:34.546753 10960 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:35799
I20260925 10:56:34.546789 11093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:35799 every 8 connection(s)
Sep 25 10:56:34 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333793, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:56:34.549191 10960 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:56:34.551950 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 10960
I20260925 10:56:34.552162 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:56:34.552925 31994 external_mini_cluster.cc:1428] Setting key 7230ad76251d800b92d6a1f86640a872
Sep 25 10:56:34 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333793, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:56:34.608343 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:34.558733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41231 (local address 127.31.62.129:56075)
0925 10:56:34.581260 (+ 22527us) negotiation.cc:107] Waiting for socket to connect
0925 10:56:34.581332 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0925 10:56:34.584222 (+ 2890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:56:34.589365 (+ 5143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:56:34.589382 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:56:34.590160 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:56:34.591177 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:34.591207 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:34.593160 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:34.593166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:34.595282 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:34.595298 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:34.596099 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:34.596105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:34.596363 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:34.597189 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:56:34.597218 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:56:34.599701 (+ 2483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:56:34.602888 (+ 3187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:56:34.602897 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:56:34.602918 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:56:34.603575 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:56:34.604044 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:56:34.604049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:56:34.604052 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:56:34.604389 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:56:34.605230 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:56:34.605239 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:56:34.605749 (+ 510us) client_negotiation.cc:769] Sending connection context
0925 10:56:34.606091 (+ 342us) client_negotiation.cc:240] Negotiation successful
0925 10:56:34.606751 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22059,"thread_start_us":29037,"threads_started":1}
I20260925 10:56:34.609870 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:34.559407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:56075 (local address 127.31.62.190:41231)
0925 10:56:34.580354 (+ 20947us) server_negotiation.cc:206] Beginning negotiation
0925 10:56:34.580362 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:56:34.584875 (+ 4513us) server_negotiation.cc:407] Connection header received
0925 10:56:34.584967 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:56:34.584973 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:56:34.585082 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:56:34.585196 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:56:34.592064 (+ 6868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:34.592946 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:34.595451 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:34.595867 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:34.595976 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:34.600013 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:56:34.600023 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:56:34.600027 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:56:34.600074 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:56:34.602712 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:34.603722 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:34.603728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:34.603731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:34.603904 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:34.604557 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:34.604562 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:34.604565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:34.604999 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:56:34.605132 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 10:56:34.609207 (+ 4075us) server_negotiation.cc:299] Negotiation successful
0925 10:56:34.609495 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20744,"thread_start_us":20963,"threads_started":1}
I20260925 10:56:34.611369 11094 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41231
I20260925 10:56:34.611707 11094 heartbeater.cc:464] Registering TS with master...
I20260925 10:56:34.612450 11094 heartbeater.cc:511] Master 127.31.62.190:41231 requested a full tablet report, sending...
I20260925 10:56:34.614938 10880 ts_manager.cc:194] Registered new tserver with Master: 442bc0f447c146cfa3324e36457ec97f (127.31.62.129:35799)
I20260925 10:56:34.617151 10880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:56075
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
W20260925 10:56:34.673216 31994 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.31.62.130: real 0.117s user 0.001s sys 0.015s
I20260925 10:56:34.674217 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.62.148:35681/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41231
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:56:35.103953 11101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:56:35.104291 11101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:56:35.104426 11101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:56:35.112538 11101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:56:35.112705 11101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:56:35.112910 11101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:56:35.123883 11101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.62.148:35681/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41231
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:56:35.125365 11101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:56:35.126688 11101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:56:35.143460 11114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:56:35.143537 11113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:56:35.144291 11116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:56:35.144567 11101 server_base.cc:1034] running on GCE node
I20260925 10:56:35.145329 11101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:56:35.146389 11101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:56:35.147644 11101 hybrid_clock.cc:648] HybridClock initialized: now 1790333795147590 us; error 50 us; skew 500 ppm
I20260925 10:56:35.147974 11101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:56:35 dist-test-slave-nx7s krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333795, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:56:35.151207 11101 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:56:35.152891 11101 webserver.cc:460] Webserver started at http://127.31.62.130:42385/ using document root <none> and password file <none>
I20260925 10:56:35.153625 11101 fs_manager.cc:359] Metadata directory not provided
I20260925 10:56:35.153736 11101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:56:35.154032 11101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 10:56:35 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333795, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:56:35.171501 11101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "71b56c6008dd4f5484b13ad993124bb4"
format_stamp: "Formatted at 2026-09-25 10:56:35 on dist-test-slave-nx7s"
server_key: "82813c17afa25b4fee9db965579bb85f"
server_key_iv: "be25899b8f984cd12700d798080dd2c4"
server_key_version: "kuduclusterkey@0"
I20260925 10:56:35.172490 11101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "71b56c6008dd4f5484b13ad993124bb4"
format_stamp: "Formatted at 2026-09-25 10:56:35 on dist-test-slave-nx7s"
server_key: "82813c17afa25b4fee9db965579bb85f"
server_key_iv: "be25899b8f984cd12700d798080dd2c4"
server_key_version: "kuduclusterkey@0"
I20260925 10:56:35.179975 11101 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260925 10:56:35.198248 11123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:56:35.200003 11101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:56:35.200196 11101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "71b56c6008dd4f5484b13ad993124bb4"
format_stamp: "Formatted at 2026-09-25 10:56:35 on dist-test-slave-nx7s"
server_key: "82813c17afa25b4fee9db965579bb85f"
server_key_iv: "be25899b8f984cd12700d798080dd2c4"
server_key_version: "kuduclusterkey@0"
I20260925 10:56:35.200415 11101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:56:35.222872 11101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:56:35.227067 11101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:56:35.227361 11101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:56:35.228329 11101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:56:35.229797 11101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:56:35.229875 11101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:56:35.230368 11101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:56:35.230427 11101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:56:35.264364 11101 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:35013
I20260925 10:56:35.264374 11235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:35013 every 8 connection(s)
I20260925 10:56:35.265832 11101 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:56:35.266819 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 11101
I20260925 10:56:35.266989 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:56:35.267499 31994 external_mini_cluster.cc:1428] Setting key a8ab163d85887165c4b7934f7db19275
Sep 25 10:56:35 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333795, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:56:35.288542 11238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:35.268921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:56853 (local address 127.31.62.190:41231)
0925 10:56:35.269402 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0925 10:56:35.269407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:56:35.270757 (+ 1350us) server_negotiation.cc:407] Connection header received
0925 10:56:35.273301 (+ 2544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:56:35.273306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:56:35.273380 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:56:35.273483 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:56:35.275549 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.276362 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.278473 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.278885 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.278997 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:35.282509 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:56:35.282514 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:56:35.282518 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:56:35.282558 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:56:35.285188 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:35.286115 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:35.286120 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:35.286122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:35.286248 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:35.286697 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:35.286702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:35.286704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:35.287005 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:56:35.287141 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 10:56:35.287943 (+ 802us) server_negotiation.cc:299] Negotiation successful
0925 10:56:35.288174 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":252,"threads_started":1}
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
I20260925 10:56:35.289655 11239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:35.269647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41231 (local address 127.31.62.130:56853)
0925 10:56:35.270570 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0925 10:56:35.270630 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 10:56:35.272896 (+ 2266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:56:35.273790 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:56:35.273803 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:56:35.274417 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:56:35.275332 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:35.275362 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.276535 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.276540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:35.278306 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:35.278320 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.279142 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.279149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:35.279376 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:35.280047 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:56:35.280071 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:56:35.282242 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:56:35.285380 (+ 3138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:56:35.285390 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:56:35.285406 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:56:35.285945 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:56:35.286359 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:56:35.286364 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:56:35.286366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:56:35.286567 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:56:35.287136 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:56:35.287145 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:56:35.287525 (+ 380us) client_negotiation.cc:769] Sending connection context
0925 10:56:35.287801 (+ 276us) client_negotiation.cc:240] Negotiation successful
0925 10:56:35.288358 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":377,"threads_started":1}
I20260925 10:56:35.291831 11236 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
I20260925 10:56:35.292219 11236 heartbeater.cc:464] Registering TS with master...
I20260925 10:56:35.293040 11236 heartbeater.cc:511] Master 127.31.62.190:41231 requested a full tablet report, sending...
I20260925 10:56:35.294739 10880 ts_manager.cc:194] Registered new tserver with Master: 71b56c6008dd4f5484b13ad993124bb4 (127.31.62.130:35013)
I20260925 10:56:35.295806 10880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:56853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:56:35.321292 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.62.148:35681/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41231
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:56:35.621171 11094 heartbeater.cc:503] Master 127.31.62.190:41231 was elected leader, sending a full tablet report...
W20260925 10:56:35.664477 11242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:56:35.664920 11242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:56:35.665123 11242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:56:35.672770 11242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:56:35.672945 11242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:56:35.673117 11242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:56:35.683662 11242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.62.148:35681/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41231
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:56:35.685065 11242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:56:35.686563 11242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:56:35.700706 11247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:56:35.700667 11250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:56:35.700745 11248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:56:35.701763 11242 server_base.cc:1034] running on GCE node
I20260925 10:56:35.702430 11242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:56:35.703397 11242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:56:35.704586 11242 hybrid_clock.cc:648] HybridClock initialized: now 1790333795704536 us; error 45 us; skew 500 ppm
I20260925 10:56:35.704954 11242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:56:35 dist-test-slave-nx7s krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333795, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:56:35.708060 11242 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:56:35.709681 11242 webserver.cc:460] Webserver started at http://127.31.62.131:38251/ using document root <none> and password file <none>
I20260925 10:56:35.710372 11242 fs_manager.cc:359] Metadata directory not provided
I20260925 10:56:35.710491 11242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:56:35.710778 11242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 10:56:35 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333795, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:56:35.726655 11242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "603d96a4d1ae468ca83c8e6f8da0db07"
format_stamp: "Formatted at 2026-09-25 10:56:35 on dist-test-slave-nx7s"
server_key: "9bed5e2fe2ab97334cc010cbf32dd95a"
server_key_iv: "529801b5d7f8a2fc8d20d378c2acc596"
server_key_version: "kuduclusterkey@0"
I20260925 10:56:35.727512 11242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "603d96a4d1ae468ca83c8e6f8da0db07"
format_stamp: "Formatted at 2026-09-25 10:56:35 on dist-test-slave-nx7s"
server_key: "9bed5e2fe2ab97334cc010cbf32dd95a"
server_key_iv: "529801b5d7f8a2fc8d20d378c2acc596"
server_key_version: "kuduclusterkey@0"
I20260925 10:56:35.733129 11242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 10:56:35.750584 11257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:56:35.752559 11242 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 10:56:35.752738 11242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "603d96a4d1ae468ca83c8e6f8da0db07"
format_stamp: "Formatted at 2026-09-25 10:56:35 on dist-test-slave-nx7s"
server_key: "9bed5e2fe2ab97334cc010cbf32dd95a"
server_key_iv: "529801b5d7f8a2fc8d20d378c2acc596"
server_key_version: "kuduclusterkey@0"
I20260925 10:56:35.752993 11242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:56:35.780593 11242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:56:35.785528 11242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:56:35.785851 11242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:56:35.786892 11242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:56:35.788280 11242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:56:35.788360 11242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:56:35.788491 11242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:56:35.788550 11242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:56:35.821727 11242 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:43695
I20260925 10:56:35.821753 11369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:43695 every 8 connection(s)
I20260925 10:56:35.823017 11242 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:56:35.823866 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 11242
I20260925 10:56:35.824016 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:56:35.824486 31994 external_mini_cluster.cc:1428] Setting key b1c77405c881bd1966ea3ae1d907f370
Sep 25 10:56:35 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333795, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:56:35.843292 11372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:35.826144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:43879 (local address 127.31.62.190:41231)
0925 10:56:35.826790 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0925 10:56:35.826794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:56:35.827757 (+ 963us) server_negotiation.cc:407] Connection header received
0925 10:56:35.829992 (+ 2235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:56:35.829996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:56:35.830075 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:56:35.830178 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:56:35.831623 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.832347 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.833924 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.834321 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.834412 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:35.837654 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:56:35.837659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:56:35.837662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:56:35.837699 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:56:35.840309 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:35.841078 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:35.841082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:35.841085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:35.841243 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:35.841693 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:35.841696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:35.841698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:35.841986 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:56:35.842089 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 10:56:35.842857 (+ 768us) server_negotiation.cc:299] Negotiation successful
0925 10:56:35.843023 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":260,"threads_started":1}
I20260925 10:56:35.844514 11373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:35.826731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41231 (local address 127.31.62.131:43879)
0925 10:56:35.827578 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0925 10:56:35.827641 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0925 10:56:35.829734 (+ 2093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:56:35.830371 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:56:35.830380 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:56:35.830844 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:56:35.831469 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:35.831490 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.832491 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.832495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:35.833797 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:35.833809 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.834537 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.834542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:35.834745 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:35.835376 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:56:35.835403 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:56:35.837438 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:56:35.840454 (+ 3016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:56:35.840461 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:56:35.840478 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:56:35.840971 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:56:35.841362 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:56:35.841366 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:56:35.841368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:56:35.841576 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:56:35.842100 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:56:35.842107 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:56:35.842449 (+ 342us) client_negotiation.cc:769] Sending connection context
0925 10:56:35.842797 (+ 348us) client_negotiation.cc:240] Negotiation successful
0925 10:56:35.843273 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":299,"threads_started":1}
I20260925 10:56:35.846112 11370 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41231
I20260925 10:56:35.846448 11370 heartbeater.cc:464] Registering TS with master...
I20260925 10:56:35.847198 11370 heartbeater.cc:511] Master 127.31.62.190:41231 requested a full tablet report, sending...
I20260925 10:56:35.848802 10880 ts_manager.cc:194] Registered new tserver with Master: 603d96a4d1ae468ca83c8e6f8da0db07 (127.31.62.131:43695)
I20260925 10:56:35.849689 10880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:43879
I20260925 10:56:35.853036 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:56:35.872267 11372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:35.858354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36804 (local address 127.31.62.190:41231)
0925 10:56:35.858583 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0925 10:56:35.858590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:56:35.858893 (+ 303us) server_negotiation.cc:407] Connection header received
0925 10:56:35.859164 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:56:35.859170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:56:35.859261 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:56:35.859349 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:56:35.860822 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.861625 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.863052 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.863473 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.863600 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:35.866303 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:56:35.866310 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:56:35.866314 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:56:35.866355 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:56:35.869263 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:35.870169 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:35.870174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:35.870177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:35.870349 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:35.870767 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:35.870771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:35.870774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:35.871068 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:56:35.871306 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0925 10:56:35.871705 (+ 399us) server_negotiation.cc:299] Negotiation successful
0925 10:56:35.871950 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260925 10:56:35.877187 10880 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:56:35.885017 10880 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:56:35.920782 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:35.899557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38730 (local address 127.31.62.129:35799)
0925 10:56:35.910967 (+ 11410us) server_negotiation.cc:206] Beginning negotiation
0925 10:56:35.910977 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:56:35.911011 (+ 34us) server_negotiation.cc:407] Connection header received
0925 10:56:35.911124 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:56:35.911133 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:56:35.911416 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:56:35.912066 (+ 650us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:56:35.913381 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.915004 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.917828 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.919423 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.919565 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:35.919838 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0925 10:56:35.920236 (+ 398us) server_negotiation.cc:299] Negotiation successful
0925 10:56:35.920429 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11200,"thread_start_us":11758,"threads_started":1}
I20260925 10:56:35.920827 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:35.899406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:35799 (local address 127.0.0.1:38730)
0925 10:56:35.900435 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0925 10:56:35.900490 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 10:56:35.900696 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:56:35.911743 (+ 11047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:56:35.911752 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:56:35.911831 (+ 79us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:56:35.913187 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:35.913200 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.915171 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.915178 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:35.917641 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:35.917660 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.919678 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.919685 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:35.919967 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:35.920021 (+ 54us) client_negotiation.cc:769] Sending connection context
0925 10:56:35.920147 (+ 126us) client_negotiation.cc:240] Negotiation successful
0925 10:56:35.920420 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"mutex_wait_us":45,"spinlock_wait_cycles":5888,"thread_start_us":981,"threads_started":1}
I20260925 10:56:35.921458 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:35.900174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45226 (local address 127.31.62.131:43695)
0925 10:56:35.910602 (+ 10428us) server_negotiation.cc:206] Beginning negotiation
0925 10:56:35.910612 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:56:35.910656 (+ 44us) server_negotiation.cc:407] Connection header received
0925 10:56:35.910789 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:56:35.910799 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:56:35.911139 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:56:35.912956 (+ 1817us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:56:35.913268 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.914597 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.917903 (+ 3306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.919497 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.920520 (+ 1023us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:35.920782 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0925 10:56:35.920898 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 10:56:35.921074 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10187,"thread_start_us":9983,"threads_started":1}
I20260925 10:56:35.921804 11385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:35.899837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:43695 (local address 127.0.0.1:45226)
0925 10:56:35.905864 (+ 6027us) negotiation.cc:107] Waiting for socket to connect
0925 10:56:35.905891 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 10:56:35.906096 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:56:35.911748 (+ 5652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:56:35.911755 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:56:35.911834 (+ 79us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:56:35.912538 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:35.912554 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.914741 (+ 2187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.914747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:35.917668 (+ 2921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:35.917685 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.919740 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.919747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:35.919984 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:35.920027 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 10:56:35.920145 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 10:56:35.920419 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1372,"mutex_wait_us":163,"spinlock_wait_cycles":39808,"thread_start_us":1236,"threads_started":1}
I20260925 10:56:35.922988 11388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:35.902067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41424 (local address 127.31.62.130:35013)
0925 10:56:35.912695 (+ 10628us) server_negotiation.cc:206] Beginning negotiation
0925 10:56:35.912704 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:56:35.912735 (+ 31us) server_negotiation.cc:407] Connection header received
0925 10:56:35.913250 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:56:35.913259 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:56:35.913567 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:56:35.913752 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:56:35.914602 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.916161 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.919538 (+ 3377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.921557 (+ 2019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.921715 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:35.921925 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0925 10:56:35.922470 (+ 545us) server_negotiation.cc:299] Negotiation successful
0925 10:56:35.922653 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10474,"thread_start_us":10354,"threads_started":1}
I20260925 10:56:35.923605 11387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:35.901512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:35013 (local address 127.0.0.1:41424)
0925 10:56:35.912656 (+ 11144us) negotiation.cc:107] Waiting for socket to connect
0925 10:56:35.912675 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:56:35.912787 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:56:35.914039 (+ 1252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:56:35.914044 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:56:35.914086 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:56:35.914413 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:35.914424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.916354 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.916360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:35.919283 (+ 2923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:35.919300 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:35.922072 (+ 2772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:35.922080 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:35.922261 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:35.922308 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 10:56:35.923111 (+ 803us) client_negotiation.cc:240] Negotiation successful
0925 10:56:35.923271 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11001,"mutex_wait_us":56,"thread_start_us":11355,"threads_started":1}
I20260925 10:56:35.925664 11028 tablet_service.cc:1467] Processing CreateTablet for tablet d8c5a8191a6241f8a749a314d37fddab (DEFAULT_TABLE table=test-table [id=8b651075daba4e9e865084306409cbaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:56:35.925814 11305 tablet_service.cc:1467] Processing CreateTablet for tablet d8c5a8191a6241f8a749a314d37fddab (DEFAULT_TABLE table=test-table [id=8b651075daba4e9e865084306409cbaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:56:35.927052 11028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c5a8191a6241f8a749a314d37fddab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:56:35.927260 11305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c5a8191a6241f8a749a314d37fddab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:56:35.927551 11171 tablet_service.cc:1467] Processing CreateTablet for tablet d8c5a8191a6241f8a749a314d37fddab (DEFAULT_TABLE table=test-table [id=8b651075daba4e9e865084306409cbaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:56:35.929076 11171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c5a8191a6241f8a749a314d37fddab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:56:35.940145 11389 tablet_bootstrap.cc:492] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f: Bootstrap starting.
I20260925 10:56:35.943276 11390 tablet_bootstrap.cc:492] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07: Bootstrap starting.
I20260925 10:56:35.943823 11389 tablet_bootstrap.cc:654] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f: Neither blocks nor log segments found. Creating new log.
I20260925 10:56:35.945438 11389 log.cc:824] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f: Log is configured to *not* fsync() on all Append() calls
I20260925 10:56:35.947404 11390 tablet_bootstrap.cc:654] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07: Neither blocks nor log segments found. Creating new log.
I20260925 10:56:35.949615 11390 log.cc:824] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07: Log is configured to *not* fsync() on all Append() calls
I20260925 10:56:35.950690 11391 tablet_bootstrap.cc:492] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4: Bootstrap starting.
I20260925 10:56:35.952600 11389 tablet_bootstrap.cc:492] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f: No bootstrap required, opened a new log
I20260925 10:56:35.952948 11389 ts_tablet_manager.cc:1397] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.003s
I20260925 10:56:35.954543 11391 tablet_bootstrap.cc:654] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4: Neither blocks nor log segments found. Creating new log.
I20260925 10:56:35.954646 11390 tablet_bootstrap.cc:492] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07: No bootstrap required, opened a new log
I20260925 10:56:35.954993 11390 ts_tablet_manager.cc:1397] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.007s
I20260925 10:56:35.955963 11391 log.cc:824] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4: Log is configured to *not* fsync() on all Append() calls
I20260925 10:56:35.963459 11390 raft_consensus.cc:348] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442bc0f447c146cfa3324e36457ec97f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35799 } } peers { permanent_uuid: "71b56c6008dd4f5484b13ad993124bb4" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35013 } } peers { permanent_uuid: "603d96a4d1ae468ca83c8e6f8da0db07" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43695 } }
I20260925 10:56:35.963905 11390 raft_consensus.cc:374] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:56:35.964027 11390 raft_consensus.cc:729] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 603d96a4d1ae468ca83c8e6f8da0db07, State: Initialized, Role: FOLLOWER
I20260925 10:56:35.964365 11391 tablet_bootstrap.cc:492] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4: No bootstrap required, opened a new log
I20260925 10:56:35.964138 11389 raft_consensus.cc:348] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442bc0f447c146cfa3324e36457ec97f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35799 } } peers { permanent_uuid: "71b56c6008dd4f5484b13ad993124bb4" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35013 } } peers { permanent_uuid: "603d96a4d1ae468ca83c8e6f8da0db07" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43695 } }
I20260925 10:56:35.964640 11389 raft_consensus.cc:374] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:56:35.964782 11389 raft_consensus.cc:729] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442bc0f447c146cfa3324e36457ec97f, State: Initialized, Role: FOLLOWER
I20260925 10:56:35.964810 11391 ts_tablet_manager.cc:1397] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.007s
I20260925 10:56:35.964735 11390 consensus_queue.cc:260] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442bc0f447c146cfa3324e36457ec97f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35799 } } peers { permanent_uuid: "71b56c6008dd4f5484b13ad993124bb4" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35013 } } peers { permanent_uuid: "603d96a4d1ae468ca83c8e6f8da0db07" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43695 } }
I20260925 10:56:35.965643 11389 consensus_queue.cc:260] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442bc0f447c146cfa3324e36457ec97f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35799 } } peers { permanent_uuid: "71b56c6008dd4f5484b13ad993124bb4" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35013 } } peers { permanent_uuid: "603d96a4d1ae468ca83c8e6f8da0db07" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43695 } }
I20260925 10:56:35.966710 11370 heartbeater.cc:503] Master 127.31.62.190:41231 was elected leader, sending a full tablet report...
I20260925 10:56:35.967252 11390 ts_tablet_manager.cc:1428] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260925 10:56:35.968914 11389 ts_tablet_manager.cc:1428] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f: Time spent starting tablet: real 0.016s user 0.007s sys 0.008s
I20260925 10:56:35.976076 11391 raft_consensus.cc:348] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442bc0f447c146cfa3324e36457ec97f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35799 } } peers { permanent_uuid: "71b56c6008dd4f5484b13ad993124bb4" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35013 } } peers { permanent_uuid: "603d96a4d1ae468ca83c8e6f8da0db07" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43695 } }
I20260925 10:56:35.976483 11391 raft_consensus.cc:374] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:56:35.976553 11391 raft_consensus.cc:729] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b56c6008dd4f5484b13ad993124bb4, State: Initialized, Role: FOLLOWER
I20260925 10:56:35.977196 11391 consensus_queue.cc:260] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442bc0f447c146cfa3324e36457ec97f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35799 } } peers { permanent_uuid: "71b56c6008dd4f5484b13ad993124bb4" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35013 } } peers { permanent_uuid: "603d96a4d1ae468ca83c8e6f8da0db07" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43695 } }
I20260925 10:56:35.978284 11236 heartbeater.cc:503] Master 127.31.62.190:41231 was elected leader, sending a full tablet report...
I20260925 10:56:35.978806 11391 ts_tablet_manager.cc:1428] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4: Time spent starting tablet: real 0.014s user 0.011s sys 0.000s
I20260925 10:56:36.001636 11397 raft_consensus.cc:482] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:56:36.001950 11397 raft_consensus.cc:504] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442bc0f447c146cfa3324e36457ec97f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35799 } } peers { permanent_uuid: "71b56c6008dd4f5484b13ad993124bb4" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35013 } } peers { permanent_uuid: "603d96a4d1ae468ca83c8e6f8da0db07" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43695 } }
I20260925 10:56:36.003808 11397 leader_election.cc:288] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442bc0f447c146cfa3324e36457ec97f (127.31.62.129:35799), 603d96a4d1ae468ca83c8e6f8da0db07 (127.31.62.131:43695)
I20260925 10:56:36.011613 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:36.004203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:45155 (local address 127.31.62.129:35799)
0925 10:56:36.004405 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 10:56:36.004410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:56:36.005822 (+ 1412us) server_negotiation.cc:407] Connection header received
0925 10:56:36.006007 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:56:36.006011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:56:36.006070 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:56:36.006150 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:56:36.006901 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.007770 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.009825 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.010651 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.010810 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:36.010889 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 10:56:36.011246 (+ 357us) server_negotiation.cc:299] Negotiation successful
0925 10:56:36.011372 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 10:56:36.011613 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:36.004370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:45353 (local address 127.31.62.131:43695)
0925 10:56:36.004568 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 10:56:36.004575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:56:36.005795 (+ 1220us) server_negotiation.cc:407] Connection header received
0925 10:56:36.005989 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:56:36.005995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:56:36.006053 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:56:36.006123 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:56:36.006765 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.007693 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.009785 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.010647 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.010819 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:36.010894 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 10:56:36.011240 (+ 346us) server_negotiation.cc:299] Negotiation successful
0925 10:56:36.011380 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 10:56:36.011855 11398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:36.004182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:43695 (local address 127.31.62.130:45353)
0925 10:56:36.005648 (+ 1466us) negotiation.cc:107] Waiting for socket to connect
0925 10:56:36.005670 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 10:56:36.005847 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:56:36.006162 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:56:36.006166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:56:36.006190 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:56:36.006620 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:36.006637 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.007857 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.007862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:36.009628 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:36.009641 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.010928 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.010935 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:36.011071 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:36.011113 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 10:56:36.011188 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 10:56:36.011337 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1339,"mutex_wait_us":24,"thread_start_us":1072,"threads_started":1}
I20260925 10:56:36.011902 11399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:36.004237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:35799 (local address 127.31.62.130:45155)
0925 10:56:36.005719 (+ 1482us) negotiation.cc:107] Waiting for socket to connect
0925 10:56:36.005735 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:56:36.005868 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:56:36.006179 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:56:36.006184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:56:36.006205 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:56:36.006683 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:36.006695 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.007913 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.007919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:36.009692 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:56:36.009707 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.010894 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.010901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:56:36.011055 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:36.011108 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 10:56:36.011199 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 10:56:36.011335 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1375,"mutex_wait_us":60,"thread_start_us":1087,"threads_started":1}
I20260925 10:56:36.013217 11325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c5a8191a6241f8a749a314d37fddab" candidate_uuid: "71b56c6008dd4f5484b13ad993124bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "603d96a4d1ae468ca83c8e6f8da0db07" is_pre_election: true
I20260925 10:56:36.013711 11325 raft_consensus.cc:2454] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71b56c6008dd4f5484b13ad993124bb4 in term 0.
I20260925 10:56:36.013722 11048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c5a8191a6241f8a749a314d37fddab" candidate_uuid: "71b56c6008dd4f5484b13ad993124bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442bc0f447c146cfa3324e36457ec97f" is_pre_election: true
I20260925 10:56:36.014302 11048 raft_consensus.cc:2454] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71b56c6008dd4f5484b13ad993124bb4 in term 0.
I20260925 10:56:36.014371 11126 leader_election.cc:302] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 603d96a4d1ae468ca83c8e6f8da0db07, 71b56c6008dd4f5484b13ad993124bb4; no voters:
I20260925 10:56:36.014758 11397 raft_consensus.cc:2781] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:56:36.014880 11397 raft_consensus.cc:482] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:56:36.014990 11397 raft_consensus.cc:3037] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:56:36.017324 11397 raft_consensus.cc:504] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442bc0f447c146cfa3324e36457ec97f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35799 } } peers { permanent_uuid: "71b56c6008dd4f5484b13ad993124bb4" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35013 } } peers { permanent_uuid: "603d96a4d1ae468ca83c8e6f8da0db07" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43695 } }
I20260925 10:56:36.017975 11397 leader_election.cc:288] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [CANDIDATE]: Term 1 election: Requested vote from peers 442bc0f447c146cfa3324e36457ec97f (127.31.62.129:35799), 603d96a4d1ae468ca83c8e6f8da0db07 (127.31.62.131:43695)
W20260925 10:56:36.018307 11237 tablet.cc:2369] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:56:36.018567 11325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c5a8191a6241f8a749a314d37fddab" candidate_uuid: "71b56c6008dd4f5484b13ad993124bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "603d96a4d1ae468ca83c8e6f8da0db07"
I20260925 10:56:36.018584 11048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c5a8191a6241f8a749a314d37fddab" candidate_uuid: "71b56c6008dd4f5484b13ad993124bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442bc0f447c146cfa3324e36457ec97f"
I20260925 10:56:36.018779 11325 raft_consensus.cc:3037] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:56:36.018780 11048 raft_consensus.cc:3037] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:56:36.022053 11048 raft_consensus.cc:2454] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71b56c6008dd4f5484b13ad993124bb4 in term 1.
I20260925 10:56:36.022364 11325 raft_consensus.cc:2454] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71b56c6008dd4f5484b13ad993124bb4 in term 1.
I20260925 10:56:36.022830 11125 leader_election.cc:302] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 442bc0f447c146cfa3324e36457ec97f, 603d96a4d1ae468ca83c8e6f8da0db07, 71b56c6008dd4f5484b13ad993124bb4; no voters:
I20260925 10:56:36.023229 11397 raft_consensus.cc:2781] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:56:36.023675 11397 raft_consensus.cc:686] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [term 1 LEADER]: Becoming Leader. State: Replica: 71b56c6008dd4f5484b13ad993124bb4, State: Running, Role: LEADER
I20260925 10:56:36.024132 11397 consensus_queue.cc:237] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442bc0f447c146cfa3324e36457ec97f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35799 } } peers { permanent_uuid: "71b56c6008dd4f5484b13ad993124bb4" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35013 } } peers { permanent_uuid: "603d96a4d1ae468ca83c8e6f8da0db07" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43695 } }
I20260925 10:56:36.029371 10878 catalog_manager.cc:5495] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71b56c6008dd4f5484b13ad993124bb4 (127.31.62.130). New cstate: current_term: 1 leader_uuid: "71b56c6008dd4f5484b13ad993124bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442bc0f447c146cfa3324e36457ec97f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71b56c6008dd4f5484b13ad993124bb4" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "603d96a4d1ae468ca83c8e6f8da0db07" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43695 } health_report { overall_health: UNKNOWN } } }
W20260925 10:56:36.074925 11371 tablet.cc:2369] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:56:36.076001 11388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:36.070294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41434 (local address 127.31.62.130:35013)
0925 10:56:36.070475 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 10:56:36.070515 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0925 10:56:36.070653 (+ 138us) server_negotiation.cc:407] Connection header received
0925 10:56:36.070874 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:56:36.070878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:56:36.070956 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:56:36.071094 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:56:36.071506 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.072197 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.073657 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.074042 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.074147 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:36.074687 (+ 540us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:56:36.075376 (+ 689us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:56:36.075539 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 10:56:36.075679 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 10:56:36.075764 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 10:56:36.085438 11048 raft_consensus.cc:1261] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f [term 1 FOLLOWER]: Refusing update from remote peer 71b56c6008dd4f5484b13ad993124bb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:56:36.085438 11325 raft_consensus.cc:1261] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07 [term 1 FOLLOWER]: Refusing update from remote peer 71b56c6008dd4f5484b13ad993124bb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:56:36.086529 11400 consensus_queue.cc:1035] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442bc0f447c146cfa3324e36457ec97f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 35799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:56:36.086973 11397 consensus_queue.cc:1035] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "603d96a4d1ae468ca83c8e6f8da0db07" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 10:56:36.093274 11095 tablet.cc:2369] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:56:36.104975 11405 mvcc.cc:205] Tried to move back new op lower bound from 7333207228755316736 to 7333207228518940672. Current Snapshot: MvccSnapshot[applied={T|T < 7333207228755316736}]
I20260925 10:56:36.122691 10879 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:56:36.123050 10879 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:56:36.127864 10879 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77787ba4951b447eb09298a0422e7aa2: Sending DeleteTablet for 3 replicas of tablet d8c5a8191a6241f8a749a314d37fddab
I20260925 10:56:36.129348 11028 tablet_service.cc:1514] Processing DeleteTablet for tablet d8c5a8191a6241f8a749a314d37fddab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:56:36 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:38730
I20260925 10:56:36.129559 11171 tablet_service.cc:1514] Processing DeleteTablet for tablet d8c5a8191a6241f8a749a314d37fddab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:56:36 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:41424
I20260925 10:56:36.130164 11304 tablet_service.cc:1514] Processing DeleteTablet for tablet d8c5a8191a6241f8a749a314d37fddab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:56:36 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:45226
I20260925 10:56:36.131230 11414 tablet_replica.cc:330] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07: stopping tablet replica
I20260925 10:56:36.131450 11305 tablet_service.cc:1514] Processing DeleteTablet for tablet d8c5a8191a6241f8a749a314d37fddab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:56:36 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:45226
I20260925 10:56:36.131791 11414 raft_consensus.cc:2229] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:56:36.132366 11414 raft_consensus.cc:2258] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:56:36.133337 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 10960
I20260925 10:56:36.134761 11414 ts_tablet_manager.cc:1905] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:56:36.142488 11414 ts_tablet_manager.cc:1918] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:56:36.142840 11414 log.cc:1196] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/d8c5a8191a6241f8a749a314d37fddab
I20260925 10:56:36.143440 11414 ts_tablet_manager.cc:1939] T d8c5a8191a6241f8a749a314d37fddab P 603d96a4d1ae468ca83c8e6f8da0db07: Deleting consensus metadata
I20260925 10:56:36.145687 10866 catalog_manager.cc:4841] TS 603d96a4d1ae468ca83c8e6f8da0db07 (127.31.62.131:43695): tablet d8c5a8191a6241f8a749a314d37fddab (table test-table [id=8b651075daba4e9e865084306409cbaa]) successfully deleted
I20260925 10:56:36.145967 11413 tablet_replica.cc:330] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4: stopping tablet replica
W20260925 10:56:36.146924 10866 catalog_manager.cc:4808] TS 603d96a4d1ae468ca83c8e6f8da0db07 (127.31.62.131:43695): delete failed for tablet d8c5a8191a6241f8a749a314d37fddab because the tablet was not found. No further retry: Not found: Tablet not found: d8c5a8191a6241f8a749a314d37fddab
I20260925 10:56:36.147221 11413 raft_consensus.cc:2229] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:56:36.150818 11412 tablet_replica.cc:330] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f: stopping tablet replica
I20260925 10:56:36.151494 11412 raft_consensus.cc:2229] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:56:36.152153 11412 raft_consensus.cc:2258] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:56:36.153545 11413 raft_consensus.cc:2258] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:56:36.155980 11413 ts_tablet_manager.cc:1905] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 25 10:56:36 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:56:36.164458 11413 ts_tablet_manager.cc:1918] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:56:36.165784 11412 ts_tablet_manager.cc:1905] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:56:36.170428 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:36.136790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.31.62.129:35799)
0925 10:56:36.136999 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0925 10:56:36.137005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:56:36.149955 (+ 12950us) server_negotiation.cc:407] Connection header received
0925 10:56:36.150212 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:56:36.150219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:56:36.150314 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:56:36.150402 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:56:36.152307 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.153454 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.155480 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.156114 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.156220 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:36.162056 (+ 5836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:56:36.162080 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:56:36.162101 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:56:36.162153 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:56:36.166184 (+ 4031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:36.167267 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:36.167279 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:36.167283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:36.167476 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:36.168004 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:36.168010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:36.168013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:36.168633 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:56:36.168989 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0925 10:56:36.169669 (+ 680us) server_negotiation.cc:299] Negotiation successful
0925 10:56:36.169945 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260925 10:56:36.172948 11089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:56:36.174813 11413 log.cc:1196] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/d8c5a8191a6241f8a749a314d37fddab
I20260925 10:56:36.175614 11413 ts_tablet_manager.cc:1939] T d8c5a8191a6241f8a749a314d37fddab P 71b56c6008dd4f5484b13ad993124bb4: Deleting consensus metadata
I20260925 10:56:36.178010 10865 catalog_manager.cc:4841] TS 71b56c6008dd4f5484b13ad993124bb4 (127.31.62.130:35013): tablet d8c5a8191a6241f8a749a314d37fddab (table test-table [id=8b651075daba4e9e865084306409cbaa]) successfully deleted
I20260925 10:56:36.344301 11412 ts_tablet_manager.cc:1918] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:56:36.344581 11412 log.cc:1196] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/d8c5a8191a6241f8a749a314d37fddab
I20260925 10:56:36.345186 11412 ts_tablet_manager.cc:1939] T d8c5a8191a6241f8a749a314d37fddab P 442bc0f447c146cfa3324e36457ec97f: Deleting consensus metadata
I20260925 10:56:36.347496 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 10960
I20260925 10:56:36.348883 10865 catalog_manager.cc:4841] TS 442bc0f447c146cfa3324e36457ec97f (127.31.62.129:35799): tablet d8c5a8191a6241f8a749a314d37fddab (table test-table [id=8b651075daba4e9e865084306409cbaa]) successfully deleted
I20260925 10:56:36.369354 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 11101
Sep 25 10:56:36 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:56:36.391522 11388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:36.370180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41450 (local address 127.31.62.130:35013)
0925 10:56:36.370357 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 10:56:36.370362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:56:36.370586 (+ 224us) server_negotiation.cc:407] Connection header received
0925 10:56:36.370795 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:56:36.370801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:56:36.370897 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:56:36.371011 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:56:36.372965 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.374319 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.376258 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.377009 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.377154 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:36.383225 (+ 6071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:56:36.383247 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:56:36.383269 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:56:36.383324 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:56:36.387689 (+ 4365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:36.388716 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:36.388727 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:36.388735 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:36.388933 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:36.389382 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:36.389388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:36.389391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:36.389895 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:56:36.390226 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0925 10:56:36.390636 (+ 410us) server_negotiation.cc:299] Negotiation successful
0925 10:56:36.390885 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 10:56:36.392382 11231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:56:36.511020 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 11101
I20260925 10:56:36.526885 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 11242
Sep 25 10:56:36 dist-test-slave-nx7s krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:56:36.542819 11418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:56:36.527757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45232 (local address 127.31.62.131:43695)
0925 10:56:36.528219 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0925 10:56:36.528223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:56:36.528237 (+ 14us) server_negotiation.cc:407] Connection header received
0925 10:56:36.528289 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:56:36.528292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:56:36.528354 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:56:36.528454 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:56:36.529823 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.530540 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.531827 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:56:36.532258 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:56:36.532366 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:56:36.537010 (+ 4644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:56:36.537026 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:56:36.537043 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:56:36.537080 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:56:36.539895 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:36.540688 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:36.540697 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:36.540699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:36.540834 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:56:36.541285 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:56:36.541288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:56:36.541290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:56:36.541666 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:56:36.541892 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0925 10:56:36.542288 (+ 396us) server_negotiation.cc:299] Negotiation successful
0925 10:56:36.542473 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":255,"threads_started":1}
I20260925 10:56:36.543526 11365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:56:36.633504 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 11242
I20260925 10:56:36.646212 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 10848
I20260925 10:56:36.647471 10910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:56:36.758232 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 10848
I20260925 10:56:36.775755 31994 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 10:56:37.111853 31994 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 10:56:37.112142 31994 mini_ranger.cc:66] Stopping Ranger...
I20260925 10:56:37.448740 31994 mini_ranger.cc:68] Stopped Ranger
2026-09-25 10:56:37.449 UTC [10434] LOG: received smart shutdown request
2026-09-25 10:56:37.452 UTC [10434] LOG: background worker "logical replication launcher" (PID 10443) exited with exit code 1
2026-09-25 10:56:37.452 UTC [10438] LOG: shutting down
2026-09-25 10:56:37.627 UTC [10434] LOG: database system is shut down
2026-09-25T10:56:37Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50935 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:56:37 dist-test-slave-nx7s krb5kdc[11429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:56:37 dist-test-slave-nx7s krb5kdc[11429](info): set up 2 sockets
Sep 25 10:56:37 dist-test-slave-nx7s krb5kdc[11429](info): commencing operation
krb5kdc: starting...
W20260925 10:56:38.757710 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:56:38 dist-test-slave-nx7s krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:56:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:56:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/rangeradmin_127.31.62.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/rangeradmin_127.31.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/rangerlookup_127.31.62.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/rangerlookup_127.31.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/HTTP_127.31.62.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/HTTP_127.31.62.148@KRBTEST.COM.keytab.
I20260925 10:56:39.068028 31994 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task9q_Uwc/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/postgres -l logfile start
2026-09-25 10:56:41.623 UTC [11459] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 10:56:41.623 UTC [11459] LOG: listening on IPv4 address "127.31.62.148", port 49987
2026-09-25 10:56:41.628 UTC [11459] LOG: listening on Unix socket "/tmp/.s.PGSQL.49987"
2026-09-25 10:56:41.649 UTC [11462] LOG: database system was shut down at 2026-09-25 10:56:40 UTC
2026-09-25 10:56:41.652 UTC [11459] LOG: database system is ready to accept connections
I20260925 10:56:42.541185 31994 mini_postgres.cc:96] Postgres bound to 49987
2026-09-25 10:56:42.557 UTC [11470] FATAL: database "slave" does not exist
127.31.62.148:49987 - accepting connections
I20260925 10:56:42.557919 31994 mini_ranger.cc:161] Starting Ranger...
I20260925 10:56:42.583396 31994 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 10:56:43.120541 31994 mini_ranger.cc:87] Created ranger Postgres database
I20260925 10:56:43.120639 31994 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-admin
2026-09-25 10:56:43,405 [I] DB FLAVOR :POSTGRES
2026-09-25 10:56:43,406 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 10:56:43,406 [I] Checking connection..
2026-09-25 10:56:43,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 10:56:43,671 [I] Checking connection passed.
2026-09-25 10:56:43,671 [I] --------- Verifying version history table ---------
2026-09-25 10:56:43,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 10:56:43,926 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 10:56:43,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 10:56:44,174 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 10:56:44,174 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 10:56:44,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 10:56:44.419 UTC [11534] WARNING: there is no transaction in progress
2026-09-25 10:56:44,425 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 10:56:44,426 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 10:56:44,670 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 10:56:44,670 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 10:56:44,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 10:56:44,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 10:56:45,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-nx7s.c.gcp-upstream.internal','N') ;"
2026-09-25 10:56:45,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 10:56:45,664 [I] Table x_portal_user does not exist in database ranger
2026-09-25 10:56:45,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 10:56:45,928 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 10:56:45,928 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 10:56:46,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 10:56:46,446 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 10:56:46,446 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 10:56:47.456 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:47.477 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:47.495 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:47.512 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:47.528 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:47.547 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:47.609 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:47.617 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:47.633 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:47.643 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:47.654 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:47.669 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:47.680 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:47.686 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:47.693 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:47.700 UTC [11642] WARNING: there is no transaction in progress
2026-09-25 10:56:48,450 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 10:56:48,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 10:56:48,704 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 10:56:48,705 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 10:56:48,961 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 10:56:48,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 10:56:49,214 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 10:56:49,214 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 10:56:49,459 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 10:56:49,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-nx7s.c.gcp-upstream.internal';"
2026-09-25 10:56:49,964 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 10:56:49,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 10:56:50,229 [I] DB_PATCHES have already been applied
I20260925 10:56:50.235180 31994 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-admin:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task9q_Uwc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 10:56:50.235275 31994 mini_ranger.cc:193] Using host: 127.31.62.148
I20260925 10:56:50.249169 31994 mini_ranger.cc:227] Ranger admin URL: http://127.31.62.148:34113
Sep 25, 2026 10:56:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 10:56:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 10:56:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 10:56:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 10:56:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 10:56:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 10:56:51 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 10:56:51.270381 31994 mini_ranger.cc:160] Time spent starting Ranger: real 8.712s user 0.005s sys 0.035s
Sep 25, 2026 10:57:12 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 10:57:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 10:57:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 10:57:12 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 10:57:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 10:57:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 10:57:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 10:57:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 10:57:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 10:57:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 10:57:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 10:57:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 10:57:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 10:57:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 10:57:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 10:57:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 10:57:12 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 10:57:13 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 10:57:13.673681 31994 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/rangerkms_127.31.62.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/rangerkms_127.31.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.62.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.62.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.62.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/HTTP_127.31.62.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.62.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/HTTP_127.31.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 10:57:13 dist-test-slave-nx7s krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333833, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 10:57:13.827327 31994 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 10:57:13.847440 31994 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 10:57:14.670737 31994 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 10:57:14.670836 31994 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-kms
I20260925 10:57:14.670874 31994 mini_ranger_kms.cc:227] Using postgres at 127.31.62.148:49987
2026-09-25 10:57:14,995 [I] DB FLAVOR :POSTGRES
2026-09-25 10:57:14,996 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 10:57:14,996 [I] Checking connection
2026-09-25 10:57:14,996 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 10:57:15,229 [I] connection success
2026-09-25 10:57:15,229 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 10:57:15,230 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 10:57:15,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 10:57:15,487 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 10:57:15,487 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 10:57:15,487 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 10:57:15,488 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.62.148:49987/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 10:57:15,737 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 10:57:16.220978 31994 mini_ranger_kms.cc:310] Created kms service
I20260925 10:57:16.389303 31994 mini_ranger_kms.cc:326] Created kudu user
I20260925 10:57:16.434166 31994 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 10:57:16 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333833, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:57:16.853823 31994 mini_ranger_kms.cc:384] Added ranger policy
I20260925 10:57:16.854151 31994 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-kms:/tmp/dist-test-task9q_Uwc/build/asan/bin/postgresql.jar:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task9q_Uwc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 10:57:16.854203 31994 mini_ranger_kms.cc:240] Using host: 127.31.62.148
I20260925 10:57:16.870669 31994 mini_ranger_kms.cc:276] Ranger KMS PID: 11822
I20260925 10:57:16.870800 31994 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.62.148:40229
Sep 25, 2026 10:57:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 10:57:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 10:57:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 10:57:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 10:57:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 10:57:18 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 10:57:18.993491 31994 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.166s user 0.060s sys 0.282s
I20260925 10:57:18.993770 31994 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 10:57:18.993948 31994 mini_ranger_kms.cc:401] 127.31.62.148:40229/kms/v1/keys
Sep 25 10:57:19 dist-test-slave-nx7s krb5kdc[11429](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790333839, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.62.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 10:57:19 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333839, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.62.148@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
Sep 25 10:57:19 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333839, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.62.148@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
Sep 25 10:57:22 dist-test-slave-nx7s krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:57:22.142594 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:37189
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:37189
--ranger_config_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.62.148:40229/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:57:22.470975 11873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:22.471274 11873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:22.471387 11873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:22.478840 11873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:57:22.478968 11873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:22.479022 11873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:22.479097 11873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:57:22.479149 11873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:57:22.489509 11873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.62.148:40229/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:37189
--ranger_config_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:37189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:22.490782 11873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:22.492112 11873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:22.503877 11878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:22.504067 11873 server_base.cc:1034] running on GCE node
W20260925 10:57:22.504005 11879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:22.503901 11881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:22.505254 11873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:22.506678 11873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:22.507894 11873 hybrid_clock.cc:648] HybridClock initialized: now 1790333842507850 us; error 37 us; skew 500 ppm
I20260925 10:57:22.508291 11873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:22 dist-test-slave-nx7s krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333842, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:22.514832 11873 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:57:22.516520 11873 webserver.cc:460] Webserver started at http://127.31.62.190:43235/ using document root <none> and password file <none>
I20260925 10:57:22.517277 11873 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:22.517416 11873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:22.517729 11873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 10:57:22 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333842, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:57:22.575448 11873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "385619c6035d47d1b44ebcd34f164fa4"
format_stamp: "Formatted at 2026-09-25 10:57:22 on dist-test-slave-nx7s"
server_key: "14c43361b362803ca00710d7c92e1ec6"
server_key_iv: "39eebe3b27206e4b1a4565ee438c8f14"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:22.576581 11873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "385619c6035d47d1b44ebcd34f164fa4"
format_stamp: "Formatted at 2026-09-25 10:57:22 on dist-test-slave-nx7s"
server_key: "14c43361b362803ca00710d7c92e1ec6"
server_key_iv: "39eebe3b27206e4b1a4565ee438c8f14"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:22.583478 11873 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260925 10:57:22.605401 11888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:22.607041 11873 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260925 10:57:22.607234 11873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "385619c6035d47d1b44ebcd34f164fa4"
format_stamp: "Formatted at 2026-09-25 10:57:22 on dist-test-slave-nx7s"
server_key: "14c43361b362803ca00710d7c92e1ec6"
server_key_iv: "39eebe3b27206e4b1a4565ee438c8f14"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:22.607467 11873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:22.622895 11873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:22.626803 11873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:22.627099 11873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:22.644445 11873 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:37189
I20260925 10:57:22.644451 11939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:37189 every 8 connection(s)
I20260925 10:57:22.645861 11873 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:57:22.649657 11940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:57:22.655738 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 11873
I20260925 10:57:22.655962 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:57:22.656754 31994 external_mini_cluster.cc:1428] Setting key 3eee194b9948aa168a2d3afde30434ec
I20260925 10:57:22.660909 11940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4: Bootstrap starting.
I20260925 10:57:22.664400 11940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4: Neither blocks nor log segments found. Creating new log.
I20260925 10:57:22.665557 11940 log.cc:824] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:22.668550 11940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4: No bootstrap required, opened a new log
Sep 25 10:57:22 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:22.676525 11940 raft_consensus.cc:348] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385619c6035d47d1b44ebcd34f164fa4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37189 } }
I20260925 10:57:22.676925 11940 raft_consensus.cc:374] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:57:22.677031 11940 raft_consensus.cc:729] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 385619c6035d47d1b44ebcd34f164fa4, State: Initialized, Role: FOLLOWER
I20260925 10:57:22.677799 11940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385619c6035d47d1b44ebcd34f164fa4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37189 } }
I20260925 10:57:22.678069 11940 raft_consensus.cc:388] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:57:22.678215 11940 raft_consensus.cc:482] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:57:22.678375 11940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:57:22.679171 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:22.658887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49310 (local address 127.31.62.190:37189)
0925 10:57:22.659642 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:22.659655 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:22.659696 (+ 41us) server_negotiation.cc:407] Connection header received
0925 10:57:22.661435 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:22.661459 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:22.661860 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:22.662195 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:22.663733 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:22.665133 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:22.666729 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:22.667381 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:22.667511 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:22.671875 (+ 4364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:22.671885 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:22.671904 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:22.671946 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:22.674957 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:22.675678 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:22.675688 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:22.675695 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:22.675846 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:22.676228 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:22.676231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:22.676234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:22.676822 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:22.677130 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:22.677669 (+ 539us) server_negotiation.cc:299] Negotiation successful
0925 10:57:22.678034 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":355,"threads_started":1}
I20260925 10:57:22.680660 11940 raft_consensus.cc:504] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385619c6035d47d1b44ebcd34f164fa4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37189 } }
I20260925 10:57:22.681186 11940 leader_election.cc:302] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 385619c6035d47d1b44ebcd34f164fa4; no voters:
I20260925 10:57:22.681777 11940 leader_election.cc:288] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:57:22.681896 11945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:57:22.682703 11945 raft_consensus.cc:686] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 1 LEADER]: Becoming Leader. State: Replica: 385619c6035d47d1b44ebcd34f164fa4, State: Running, Role: LEADER
I20260925 10:57:22.683182 11945 consensus_queue.cc:237] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385619c6035d47d1b44ebcd34f164fa4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37189 } }
I20260925 10:57:22.683785 11940 sys_catalog.cc:564] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:57:22.686069 11947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 385619c6035d47d1b44ebcd34f164fa4. Latest consensus state: current_term: 1 leader_uuid: "385619c6035d47d1b44ebcd34f164fa4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385619c6035d47d1b44ebcd34f164fa4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37189 } } }
I20260925 10:57:22.686508 11947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [sys.catalog]: This master's current role is: LEADER
I20260925 10:57:22.686887 11946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "385619c6035d47d1b44ebcd34f164fa4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385619c6035d47d1b44ebcd34f164fa4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37189 } } }
I20260925 10:57:22.687304 11946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [sys.catalog]: This master's current role is: LEADER
I20260925 10:57:22.688161 11954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:57:22.693588 11954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:57:22.694410 11940 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 10:57:22.712596 11954 catalog_manager.cc:1347] Generated new cluster ID: 0d06131a18754bb7b8a567ec543cab14
I20260925 10:57:22.712810 11954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:57:22.728240 11954 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:57:22.730398 11954 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:57:22.750272 11954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4: Generated new TSK 0
I20260925 10:57:22.751451 11954 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:57:23.506276 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.62.148:40229/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:37189
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 10:57:23 dist-test-slave-nx7s krb5kdc[11429](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790333843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 10:57:23.897405 11985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:23.897703 11985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:23.897876 11985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:23.905339 11985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:23.905464 11985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:23.905642 11985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:57:23.916453 11985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.62.148:40229/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:37189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:23.917846 11985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:23.919219 11985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:23.933075 11996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:23.933230 11998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:23.934309 11995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:23.936993 11985 server_base.cc:1034] running on GCE node
I20260925 10:57:23.937700 11985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:23.938906 11985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:23.940132 11985 hybrid_clock.cc:648] HybridClock initialized: now 1790333843940063 us; error 82 us; skew 500 ppm
I20260925 10:57:23.940450 11985 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:23 dist-test-slave-nx7s krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:23.943723 11985 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:57:23.951514 11985 webserver.cc:460] Webserver started at http://127.31.62.129:43767/ using document root <none> and password file <none>
I20260925 10:57:23.952287 11985 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:23.952412 11985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:23.952740 11985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 10:57:23 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:57:23.982810 11985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "64c4d0eb70d74aaa93c647e86f62cbfb"
format_stamp: "Formatted at 2026-09-25 10:57:23 on dist-test-slave-nx7s"
server_key: "4a84cecf1ef5eb88c3251a02082fcd86"
server_key_iv: "d492a4cbeb519eb4b73e14f4876db573"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:23.984076 11985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "64c4d0eb70d74aaa93c647e86f62cbfb"
format_stamp: "Formatted at 2026-09-25 10:57:23 on dist-test-slave-nx7s"
server_key: "4a84cecf1ef5eb88c3251a02082fcd86"
server_key_iv: "d492a4cbeb519eb4b73e14f4876db573"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:23.992621 11985 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.003s
I20260925 10:57:24.025559 12005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:24.038632 11985 fs_manager.cc:725] Time spent opening block manager: real 0.024s user 0.005s sys 0.008s
I20260925 10:57:24.038955 11985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "64c4d0eb70d74aaa93c647e86f62cbfb"
format_stamp: "Formatted at 2026-09-25 10:57:23 on dist-test-slave-nx7s"
server_key: "4a84cecf1ef5eb88c3251a02082fcd86"
server_key_iv: "d492a4cbeb519eb4b73e14f4876db573"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:24.039330 11985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:24.071125 11985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:24.075675 11985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:24.075968 11985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:24.077016 11985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:57:24.078529 11985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:57:24.078647 11985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260925 10:57:24.078794 11985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:57:24.078848 11985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 10:57:24 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:57:24.459023 11985 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:41361
I20260925 10:57:24.459065 12117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:41361 every 8 connection(s)
I20260925 10:57:24.460770 11985 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:57:24.462350 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 11985
I20260925 10:57:24.462559 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:57:24.463358 31994 external_mini_cluster.cc:1428] Setting key 60aee4e534dfc1a2e90f30282205e7ac
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
Sep 25 10:57:24 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:24.509240 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:24.465611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:50383 (local address 127.31.62.190:37189)
0925 10:57:24.482120 (+ 16509us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:24.482127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:24.486009 (+ 3882us) server_negotiation.cc:407] Connection header received
0925 10:57:24.489728 (+ 3719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:24.489735 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:24.489838 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:24.489915 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:24.492231 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:24.493424 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:24.495990 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:24.496646 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:24.496759 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:24.501309 (+ 4550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:24.501316 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:24.501322 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:24.501390 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:24.504924 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:24.505881 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:24.505888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:24.505891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:24.506049 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:24.506616 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:24.506620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:24.506624 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:24.506987 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:24.507113 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:24.508242 (+ 1129us) server_negotiation.cc:299] Negotiation successful
0925 10:57:24.508509 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16360,"thread_start_us":17779,"threads_started":1}
I20260925 10:57:24.511857 12120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:24.464589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:37189 (local address 127.31.62.129:50383)
0925 10:57:24.485798 (+ 21209us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:24.485865 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:24.489357 (+ 3492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:24.490512 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:24.490525 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:24.491139 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:24.492008 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:24.492037 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:24.493690 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:24.493695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:24.495790 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:24.495816 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:24.497131 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:24.497137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:24.497481 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:24.498251 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:24.498281 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:24.501021 (+ 2740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:24.505104 (+ 4083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:24.505113 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:24.505133 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:24.505735 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:24.506204 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:24.506208 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:24.506212 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:24.506484 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:24.507227 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:24.507235 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:24.507723 (+ 488us) client_negotiation.cc:769] Sending connection context
0925 10:57:24.509579 (+ 1856us) client_negotiation.cc:240] Negotiation successful
0925 10:57:24.510175 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20833,"thread_start_us":21295,"threads_started":1}
I20260925 10:57:24.514331 12118 heartbeater.cc:347] Connected to a master server at 127.31.62.190:37189
I20260925 10:57:24.514674 12118 heartbeater.cc:464] Registering TS with master...
I20260925 10:57:24.515596 12118 heartbeater.cc:511] Master 127.31.62.190:37189 requested a full tablet report, sending...
I20260925 10:57:24.518217 11905 ts_manager.cc:194] Registered new tserver with Master: 64c4d0eb70d74aaa93c647e86f62cbfb (127.31.62.129:41361)
I20260925 10:57:24.520188 11905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:50383
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:57:24.536163 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.62.148:40229/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:37189
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 10:57:24 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333843, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
W20260925 10:57:25.032568 12124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:25.032908 12124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:25.033066 12124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:25.040627 12124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:25.040768 12124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:25.040977 12124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:57:25.051287 12124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.62.148:40229/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:37189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:25.052666 12124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:25.054064 12124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:25.068876 12138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:25.072745 12139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:25.074148 12141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:25.074240 12124 server_base.cc:1034] running on GCE node
I20260925 10:57:25.074872 12124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:25.075917 12124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:25.077122 12124 hybrid_clock.cc:648] HybridClock initialized: now 1790333845077070 us; error 47 us; skew 500 ppm
I20260925 10:57:25.077435 12124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:25 dist-test-slave-nx7s krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333845, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:25.080698 12124 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:57:25.082428 12124 webserver.cc:460] Webserver started at http://127.31.62.130:40931/ using document root <none> and password file <none>
I20260925 10:57:25.083148 12124 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:25.083261 12124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:25.083551 12124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 10:57:25 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333845, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:57:25.102264 12124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "661bb149641346a581d2226e9dbc43e8"
format_stamp: "Formatted at 2026-09-25 10:57:25 on dist-test-slave-nx7s"
server_key: "20cd9ae0b7610d63c68251c73c18c803"
server_key_iv: "f7ad6c777cbde737b66cc4950d883a1f"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:25.103508 12124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "661bb149641346a581d2226e9dbc43e8"
format_stamp: "Formatted at 2026-09-25 10:57:25 on dist-test-slave-nx7s"
server_key: "20cd9ae0b7610d63c68251c73c18c803"
server_key_iv: "f7ad6c777cbde737b66cc4950d883a1f"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:25.110921 12124 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260925 10:57:25.129426 12148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:25.131394 12124 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 10:57:25.131664 12124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "661bb149641346a581d2226e9dbc43e8"
format_stamp: "Formatted at 2026-09-25 10:57:25 on dist-test-slave-nx7s"
server_key: "20cd9ae0b7610d63c68251c73c18c803"
server_key_iv: "f7ad6c777cbde737b66cc4950d883a1f"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:25.131984 12124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:25.151331 12124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:25.156476 12124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:25.156881 12124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:25.158229 12124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:57:25.160084 12124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:57:25.160223 12124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:25.160374 12124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:57:25.160481 12124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:25.196367 12124 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:44903
I20260925 10:57:25.196406 12260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:44903 every 8 connection(s)
I20260925 10:57:25.198082 12124 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:57:25.199000 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 12124
I20260925 10:57:25.199153 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:57:25.199683 31994 external_mini_cluster.cc:1428] Setting key 0ae7b0ca9d4b2749eca87bed1632e229
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
Sep 25 10:57:25 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333845, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:25.252243 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:25.209091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:60131 (local address 127.31.62.190:37189)
0925 10:57:25.217268 (+ 8177us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:25.217275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:25.225767 (+ 8492us) server_negotiation.cc:407] Connection header received
0925 10:57:25.228481 (+ 2714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:25.228488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:25.228599 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:25.228684 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:25.231105 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:25.232228 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:25.234777 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:25.235422 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:25.235532 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:25.244375 (+ 8843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:25.244384 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:25.244389 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:25.244438 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:25.247890 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:25.249027 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:25.249034 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:25.249038 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:25.249225 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:25.249840 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:25.249846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:25.249849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:25.250308 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:25.250449 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:25.251558 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0925 10:57:25.251820 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8029,"thread_start_us":7906,"threads_started":1}
I20260925 10:57:25.255023 12263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:25.204074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:37189 (local address 127.31.62.130:60131)
0925 10:57:25.225540 (+ 21466us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:25.225602 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:25.228073 (+ 2471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:25.229328 (+ 1255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:25.229342 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:25.229942 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:25.230853 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:25.230883 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:25.232456 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:25.232462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:25.234548 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:25.234612 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:25.235926 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:25.235933 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:25.236244 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:25.241133 (+ 4889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:25.241170 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:25.244014 (+ 2844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:25.248116 (+ 4102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:25.248127 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:25.248148 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:25.248830 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:25.249390 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:25.249395 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:25.249399 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:25.249697 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:25.250555 (+ 858us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:25.250563 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:25.251059 (+ 496us) client_negotiation.cc:769] Sending connection context
0925 10:57:25.252834 (+ 1775us) client_negotiation.cc:240] Negotiation successful
0925 10:57:25.253488 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21132,"thread_start_us":20902,"threads_started":1}
I20260925 10:57:25.257256 12261 heartbeater.cc:347] Connected to a master server at 127.31.62.190:37189
I20260925 10:57:25.257604 12261 heartbeater.cc:464] Registering TS with master...
I20260925 10:57:25.258546 12261 heartbeater.cc:511] Master 127.31.62.190:37189 requested a full tablet report, sending...
I20260925 10:57:25.260401 11905 ts_manager.cc:194] Registered new tserver with Master: 661bb149641346a581d2226e9dbc43e8 (127.31.62.130:44903)
I20260925 10:57:25.261556 11905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:60131
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:57:25.271899 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.62.148:40229/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:37189
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:57:25.524180 12118 heartbeater.cc:503] Master 127.31.62.190:37189 was elected leader, sending a full tablet report...
W20260925 10:57:25.598816 12267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:25.599125 12267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:25.599246 12267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:25.606523 12267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:25.606666 12267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:25.606819 12267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:57:25.616809 12267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.62.148:40229/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:37189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:25.618145 12267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:25.619414 12267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:25.632750 12275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:25.632714 12272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:25.632793 12273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:25.638222 12267 server_base.cc:1034] running on GCE node
I20260925 10:57:25.638870 12267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:25.639850 12267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:25.641032 12267 hybrid_clock.cc:648] HybridClock initialized: now 1790333845640985 us; error 39 us; skew 500 ppm
I20260925 10:57:25.641337 12267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:25 dist-test-slave-nx7s krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333845, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:25.644301 12267 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:57:25.645918 12267 webserver.cc:460] Webserver started at http://127.31.62.131:35921/ using document root <none> and password file <none>
I20260925 10:57:25.646658 12267 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:25.646775 12267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:25.647073 12267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 10:57:25 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333845, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:57:25.662308 12267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "6cdb92dbb0ed4f369e72507161627b6e"
format_stamp: "Formatted at 2026-09-25 10:57:25 on dist-test-slave-nx7s"
server_key: "1c6f088d115cb674c1d214bfe30b3c79"
server_key_iv: "c67aebe70c3fa54c28b460d036d86161"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:25.663244 12267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "6cdb92dbb0ed4f369e72507161627b6e"
format_stamp: "Formatted at 2026-09-25 10:57:25 on dist-test-slave-nx7s"
server_key: "1c6f088d115cb674c1d214bfe30b3c79"
server_key_iv: "c67aebe70c3fa54c28b460d036d86161"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:25.668608 12267 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 10:57:25.683075 12282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:25.684640 12267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:57:25.684813 12267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "6cdb92dbb0ed4f369e72507161627b6e"
format_stamp: "Formatted at 2026-09-25 10:57:25 on dist-test-slave-nx7s"
server_key: "1c6f088d115cb674c1d214bfe30b3c79"
server_key_iv: "c67aebe70c3fa54c28b460d036d86161"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:25.685068 12267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:25.698933 12267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:25.702448 12267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:25.702749 12267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:25.703899 12267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:57:25.705286 12267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:57:25.705355 12267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:25.705457 12267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:57:25.705505 12267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:25.735009 12267 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:35015
I20260925 10:57:25.735052 12394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:35015 every 8 connection(s)
I20260925 10:57:25.736315 12267 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:57:25.745294 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 12267
I20260925 10:57:25.745469 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:57:25.745945 31994 external_mini_cluster.cc:1428] Setting key 364522a73b769c5eebf83e95c9211653
Sep 25 10:57:25 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333845, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:25.754750 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:25.738884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:52605 (local address 127.31.62.190:37189)
0925 10:57:25.739055 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:25.739059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:25.740176 (+ 1117us) server_negotiation.cc:407] Connection header received
0925 10:57:25.741945 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:25.741951 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:25.742043 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:25.742131 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:25.743555 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:25.744296 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:25.745999 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:25.746426 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:25.746535 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:25.749981 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:25.749986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:25.749989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:25.750021 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:25.752227 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:25.752911 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:25.752915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:25.752917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:25.753038 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:25.753382 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:25.753385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:25.753386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:25.753643 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:25.753758 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:25.754336 (+ 578us) server_negotiation.cc:299] Negotiation successful
0925 10:57:25.754499 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 10:57:25.755661 12397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:25.739348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:37189 (local address 127.31.62.131:52605)
0925 10:57:25.740004 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:25.740041 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:25.741679 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:25.742381 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:25.742389 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:25.742789 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:25.743387 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:25.743404 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:25.744440 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:25.744444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:25.745856 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:25.745875 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:25.746636 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:25.746643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:25.746888 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:25.747553 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:25.747575 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:25.749792 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:25.752345 (+ 2553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:25.752352 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:25.752366 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:25.752797 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:25.753116 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:25.753119 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:25.753121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:25.753302 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:25.753735 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:25.753740 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:25.754031 (+ 291us) client_negotiation.cc:769] Sending connection context
0925 10:57:25.754266 (+ 235us) client_negotiation.cc:240] Negotiation successful
0925 10:57:25.754633 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":262,"threads_started":1}
I20260925 10:57:25.756955 12395 heartbeater.cc:347] Connected to a master server at 127.31.62.190:37189
I20260925 10:57:25.757246 12395 heartbeater.cc:464] Registering TS with master...
I20260925 10:57:25.757874 12395 heartbeater.cc:511] Master 127.31.62.190:37189 requested a full tablet report, sending...
I20260925 10:57:25.759176 11905 ts_manager.cc:194] Registered new tserver with Master: 6cdb92dbb0ed4f369e72507161627b6e (127.31.62.131:35015)
I20260925 10:57:25.760071 11905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:52605
I20260925 10:57:25.761437 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:57:25.763381 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 11985
Sep 25 10:57:25 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:57:25.779036 12399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:25.764124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49566 (local address 127.31.62.129:41361)
0925 10:57:25.764639 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:25.764645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:25.764671 (+ 26us) server_negotiation.cc:407] Connection header received
0925 10:57:25.764801 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:25.764808 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:25.765056 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:25.765223 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:25.766582 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:25.767553 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:25.769006 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:25.769559 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:25.769665 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:25.773792 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:25.773805 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:25.773822 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:25.773858 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:25.776607 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:25.777303 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:25.777311 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:25.777314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:25.777452 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:25.777773 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:25.777775 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:25.777777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:25.778094 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:25.778327 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:25.778550 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 10:57:25.778758 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":225,"threads_started":1}
I20260925 10:57:25.780336 12113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:25.864352 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 11985
I20260925 10:57:25.876748 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 12124
Sep 25 10:57:25 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:57:25.893111 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:25.877728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57652 (local address 127.31.62.130:44903)
0925 10:57:25.878349 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:25.878357 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:25.878382 (+ 25us) server_negotiation.cc:407] Connection header received
0925 10:57:25.878438 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:25.878444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:25.878693 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:25.878818 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:25.880167 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:25.881197 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:25.882664 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:25.883275 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:25.883420 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:25.887815 (+ 4395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:25.887827 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:25.887843 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:25.887885 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:25.890543 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:25.891200 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:25.891207 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:25.891210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:25.891348 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:25.891669 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:25.891672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:25.891675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:25.892010 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:25.892219 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:25.892521 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 10:57:25.892720 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":368,"threads_started":1}
I20260925 10:57:25.894358 12256 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:25.975696 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 12124
I20260925 10:57:25.989652 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 12267
Sep 25 10:57:25 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:57:26.006192 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:25.990659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34954 (local address 127.31.62.131:35015)
0925 10:57:25.991291 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:25.991298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:25.991326 (+ 28us) server_negotiation.cc:407] Connection header received
0925 10:57:25.991383 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:25.991389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:25.991626 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:25.991771 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:25.993132 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:25.994197 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:25.995488 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:25.996059 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:25.996173 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:26.000910 (+ 4737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:26.000924 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:26.000939 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:26.000981 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:26.003583 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:26.004324 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:26.004331 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:26.004333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:26.004466 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:26.004793 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:26.004796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:26.004798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:26.005196 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:26.005446 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:26.005696 (+ 250us) server_negotiation.cc:299] Negotiation successful
0925 10:57:26.005906 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":361,"threads_started":1}
I20260925 10:57:26.007479 12390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:26.089205 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 12267
I20260925 10:57:26.101559 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 11873
I20260925 10:57:26.102811 11935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:26.208779 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 11873
I20260925 10:57:26.226253 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:37189
--webserver_interface=127.31.62.190
--webserver_port=43235
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:37189
--ranger_config_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.62.148:40229/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.62.148:40229/kms with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:57:26.545353 12406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:26.545655 12406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:26.545773 12406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:26.553027 12406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:57:26.553148 12406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:26.553202 12406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:26.553278 12406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:57:26.553342 12406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:57:26.563558 12406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.62.148:40229/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:37189
--ranger_config_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:37189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=43235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:26.564913 12406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:26.566308 12406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:26.578122 12412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:26.578174 12411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:26.578145 12414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:26.579695 12406 server_base.cc:1034] running on GCE node
I20260925 10:57:26.580490 12406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:26.581985 12406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:26.583182 12406 hybrid_clock.cc:648] HybridClock initialized: now 1790333846583134 us; error 38 us; skew 500 ppm
I20260925 10:57:26.583529 12406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:26 dist-test-slave-nx7s krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333846, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:26.586658 12406 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:57:26.588097 12406 webserver.cc:460] Webserver started at http://127.31.62.190:43235/ using document root <none> and password file <none>
I20260925 10:57:26.588735 12406 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:26.588835 12406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 10:57:26 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333846, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:57:26.654217 12406 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 10:57:26.657328 12422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:26.658854 12406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:57:26.659020 12406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "385619c6035d47d1b44ebcd34f164fa4"
format_stamp: "Formatted at 2026-09-25 10:57:22 on dist-test-slave-nx7s"
server_key: "14c43361b362803ca00710d7c92e1ec6"
server_key_iv: "39eebe3b27206e4b1a4565ee438c8f14"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:26.659991 12406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:26.675781 12406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:26.679947 12406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:26.680230 12406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:26.697674 12406 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:37189
I20260925 10:57:26.697700 12473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:37189 every 8 connection(s)
I20260925 10:57:26.699210 12406 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:57:26.706161 12474 sys_catalog.cc:263] Verifying existing consensus state
I20260925 10:57:26.708400 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 12406
I20260925 10:57:26.709435 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.62.148:40229/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:41361
--local_ip_for_outbound_sockets=127.31.62.129
--tserver_master_addrs=127.31.62.190:37189
--webserver_port=43767
--webserver_interface=127.31.62.129
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.62.148:40229/kms with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:57:26.710182 12474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4: Bootstrap starting.
I20260925 10:57:26.730553 12474 log.cc:824] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:26.739840 12474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 10:57:26.740352 12474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4: Bootstrap complete.
I20260925 10:57:26.749054 12474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385619c6035d47d1b44ebcd34f164fa4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37189 } }
I20260925 10:57:26.749895 12474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 385619c6035d47d1b44ebcd34f164fa4, State: Initialized, Role: FOLLOWER
I20260925 10:57:26.750519 12474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385619c6035d47d1b44ebcd34f164fa4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37189 } }
I20260925 10:57:26.750723 12474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:57:26.750834 12474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:57:26.750990 12474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 1 FOLLOWER]: Advancing to term 2
I20260925 10:57:26.754804 12474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385619c6035d47d1b44ebcd34f164fa4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37189 } }
I20260925 10:57:26.755318 12474 leader_election.cc:302] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 385619c6035d47d1b44ebcd34f164fa4; no voters:
I20260925 10:57:26.755880 12474 leader_election.cc:288] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 10:57:26.756017 12478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 10:57:26.757169 12478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [term 2 LEADER]: Becoming Leader. State: Replica: 385619c6035d47d1b44ebcd34f164fa4, State: Running, Role: LEADER
I20260925 10:57:26.758041 12474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:57:26.757925 12478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385619c6035d47d1b44ebcd34f164fa4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37189 } }
I20260925 10:57:26.760411 12479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "385619c6035d47d1b44ebcd34f164fa4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385619c6035d47d1b44ebcd34f164fa4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37189 } } }
I20260925 10:57:26.760787 12479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [sys.catalog]: This master's current role is: LEADER
I20260925 10:57:26.761173 12480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 385619c6035d47d1b44ebcd34f164fa4. Latest consensus state: current_term: 2 leader_uuid: "385619c6035d47d1b44ebcd34f164fa4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385619c6035d47d1b44ebcd34f164fa4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37189 } } }
I20260925 10:57:26.761559 12480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4 [sys.catalog]: This master's current role is: LEADER
I20260925 10:57:26.766467 12484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:57:26.772462 12484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:57:26.773943 12484 catalog_manager.cc:1259] Loaded cluster ID: 0d06131a18754bb7b8a567ec543cab14
I20260925 10:57:26.774139 12484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:57:26.778237 12484 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:57:26.779804 12484 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4: Loaded TSK: 0
I20260925 10:57:26.781556 12484 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:57:26.786270 12474 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260925 10:57:27.142336 12476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:27.142848 12476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:27.143064 12476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:27.153559 12476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:27.153688 12476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:27.153844 12476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:57:27.167600 12476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.62.148:40229/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:41361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=43767
--tserver_master_addrs=127.31.62.190:37189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:27.169404 12476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:27.170780 12476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:27.195870 12511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:27.197707 12512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:27.203282 12476 server_base.cc:1034] running on GCE node
W20260925 10:57:27.203675 12514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:27.204619 12476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:27.207294 12476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:27.208581 12476 hybrid_clock.cc:648] HybridClock initialized: now 1790333847208508 us; error 63 us; skew 500 ppm
I20260925 10:57:27.209131 12476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:27 dist-test-slave-nx7s krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333847, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:27.213495 12476 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:57:27.222606 12476 webserver.cc:460] Webserver started at http://127.31.62.129:43767/ using document root <none> and password file <none>
I20260925 10:57:27.223692 12476 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:27.223883 12476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 10:57:27 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333847, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:57:27.267108 12476 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.003s sys 0.012s
I20260925 10:57:27.271970 12522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:27.273878 12476 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260925 10:57:27.274159 12476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "64c4d0eb70d74aaa93c647e86f62cbfb"
format_stamp: "Formatted at 2026-09-25 10:57:23 on dist-test-slave-nx7s"
server_key: "4a84cecf1ef5eb88c3251a02082fcd86"
server_key_iv: "d492a4cbeb519eb4b73e14f4876db573"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:27.275836 12476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:27.295873 12476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:27.301626 12476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:27.302032 12476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:27.307778 12476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:57:27.309916 12476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:57:27.310055 12476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:27.310180 12476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:57:27.310259 12476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:27.397457 12476 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:41361
I20260925 10:57:27.397488 12635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:41361 every 8 connection(s)
I20260925 10:57:27.399456 12476 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:57:27.405597 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 12476
I20260925 10:57:27.406417 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.62.148:40229/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:44903
--local_ip_for_outbound_sockets=127.31.62.130
--tserver_master_addrs=127.31.62.190:37189
--webserver_port=40931
--webserver_interface=127.31.62.130
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.62.148:40229/kms with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 10:57:27 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333847, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:27.430982 12639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:27.402846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:37189 (local address 127.31.62.129:52179)
0925 10:57:27.403646 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:27.403686 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:27.405586 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:27.407466 (+ 1880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:27.407478 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:27.408110 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:27.408906 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:27.408932 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:27.411332 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:27.411337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:27.413165 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:27.413190 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:27.414906 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:27.414915 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:27.415212 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:27.416749 (+ 1537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:27.416785 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:27.421354 (+ 4569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:27.426172 (+ 4818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:27.426186 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:27.426206 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:27.426916 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:27.427529 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:27.427534 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:27.427538 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:27.427829 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:27.428919 (+ 1090us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:27.428927 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:27.429300 (+ 373us) client_negotiation.cc:769] Sending connection context
0925 10:57:27.429553 (+ 253us) client_negotiation.cc:240] Negotiation successful
0925 10:57:27.429974 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":372,"threads_started":1}
I20260925 10:57:27.431599 12638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:27.402582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:52179 (local address 127.31.62.190:37189)
0925 10:57:27.403295 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:27.403313 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:27.403821 (+ 508us) server_negotiation.cc:407] Connection header received
0925 10:57:27.406280 (+ 2459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:27.406300 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:27.406782 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:27.407178 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:27.409145 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:27.411133 (+ 1988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:27.413366 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:27.414398 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:27.414591 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:27.421765 (+ 7174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:27.421778 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:27.421802 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:27.421854 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:27.425946 (+ 4092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:27.427139 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:27.427149 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:27.427159 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:27.427379 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:27.427977 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:27.427982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:27.427985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:27.428750 (+ 765us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:27.428981 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:27.429868 (+ 887us) server_negotiation.cc:299] Negotiation successful
0925 10:57:27.430322 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":275,"threads_started":1}
I20260925 10:57:27.434965 12636 heartbeater.cc:347] Connected to a master server at 127.31.62.190:37189
I20260925 10:57:27.435299 12636 heartbeater.cc:464] Registering TS with master...
I20260925 10:57:27.436194 12636 heartbeater.cc:511] Master 127.31.62.190:37189 requested a full tablet report, sending...
W20260925 10:57:27.439477 12636 heartbeater.cc:644] Failed to heartbeat to 127.31.62.190:37189 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 10:57:27.439707 12636 heartbeater.cc:464] Registering TS with master...
I20260925 10:57:27.439987 12636 heartbeater.cc:511] Master 127.31.62.190:37189 requested a full tablet report, sending...
I20260925 10:57:27.441135 12636 heartbeater.cc:464] Registering TS with master...
I20260925 10:57:27.441423 12636 heartbeater.cc:511] Master 127.31.62.190:37189 requested a full tablet report, sending...
W20260925 10:57:27.442498 12636 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 10:57:27.823565 12640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:27.823988 12640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:27.824159 12640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:27.835793 12640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:27.835951 12640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:27.836131 12640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:57:27.852020 12640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.62.148:40229/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:44903
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=40931
--tserver_master_addrs=127.31.62.190:37189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:27.853884 12640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:27.855787 12640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:27.872511 12655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:27.873276 12657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:27.874487 12654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:27.875213 12640 server_base.cc:1034] running on GCE node
I20260925 10:57:27.876034 12640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:27.877394 12640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:27.878667 12640 hybrid_clock.cc:648] HybridClock initialized: now 1790333847878595 us; error 69 us; skew 500 ppm
I20260925 10:57:27.879079 12640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:27 dist-test-slave-nx7s krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333847, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:27.890558 12640 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:57:27.892813 12640 webserver.cc:460] Webserver started at http://127.31.62.130:40931/ using document root <none> and password file <none>
I20260925 10:57:27.893882 12640 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:27.894066 12640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 10:57:27 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333847, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:57:27.917582 12640 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 10:57:27.921416 12665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:27.925915 12640 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.001s
I20260925 10:57:27.926204 12640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "661bb149641346a581d2226e9dbc43e8"
format_stamp: "Formatted at 2026-09-25 10:57:25 on dist-test-slave-nx7s"
server_key: "20cd9ae0b7610d63c68251c73c18c803"
server_key_iv: "f7ad6c777cbde737b66cc4950d883a1f"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:27.927690 12640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:27.977159 12640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:27.982651 12640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:27.983043 12640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:27.984346 12640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:57:27.986385 12640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:57:27.986572 12640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260925 10:57:27.986738 12640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:57:27.986876 12640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:28.031523 12640 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:44903
I20260925 10:57:28.031592 12779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:44903 every 8 connection(s)
I20260925 10:57:28.033528 12640 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:57:28.034334 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 12640
I20260925 10:57:28.035202 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.62.148:40229/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:35015
--local_ip_for_outbound_sockets=127.31.62.131
--tserver_master_addrs=127.31.62.190:37189
--webserver_port=35921
--webserver_interface=127.31.62.131
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.62.148:40229/kms with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 10:57:28 dist-test-slave-nx7s krb5kdc[11429](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790333848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 10:57:28 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333847, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:28.078924 12782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:28.043547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:37189 (local address 127.31.62.130:54199)
0925 10:57:28.044476 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:28.044531 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:28.047564 (+ 3033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:28.050429 (+ 2865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:28.050442 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:28.051047 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:28.051987 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:28.052014 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.053556 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.053562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:28.055535 (+ 1973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:28.055549 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.056762 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.056769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:28.057103 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:28.066011 (+ 8908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:28.066052 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:28.069702 (+ 3650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:28.073924 (+ 4222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:28.073935 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:28.073955 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:28.074585 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:28.075055 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:28.075060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:28.075064 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:28.075335 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:28.076060 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:28.076068 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:28.076551 (+ 483us) client_negotiation.cc:769] Sending connection context
0925 10:57:28.076881 (+ 330us) client_negotiation.cc:240] Negotiation successful
0925 10:57:28.077479 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":703,"threads_started":1}
I20260925 10:57:28.080124 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:28.044213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:54199 (local address 127.31.62.190:37189)
0925 10:57:28.049650 (+ 5437us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:28.049656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:28.049677 (+ 21us) server_negotiation.cc:407] Connection header received
0925 10:57:28.049741 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:28.049747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:28.049834 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:28.049948 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:28.052226 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.053357 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.055713 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.056331 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.056455 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:28.070040 (+ 13585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:28.070048 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:28.070054 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:28.070108 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:28.073708 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:28.074755 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:28.074760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:28.074764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:28.074916 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:28.075482 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:28.075487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:28.075491 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:28.075849 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:28.075977 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:28.079531 (+ 3554us) server_negotiation.cc:299] Negotiation successful
0925 10:57:28.079777 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5299,"thread_start_us":5168,"threads_started":1}
I20260925 10:57:28.082545 12780 heartbeater.cc:347] Connected to a master server at 127.31.62.190:37189
I20260925 10:57:28.082878 12780 heartbeater.cc:464] Registering TS with master...
I20260925 10:57:28.083778 12780 heartbeater.cc:511] Master 127.31.62.190:37189 requested a full tablet report, sending...
I20260925 10:57:28.086066 12439 ts_manager.cc:194] Registered new tserver with Master: 661bb149641346a581d2226e9dbc43e8 (127.31.62.130:44903)
I20260925 10:57:28.088259 12439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:54199
I20260925 10:57:28.442647 12636 heartbeater.cc:464] Registering TS with master...
I20260925 10:57:28.443095 12636 heartbeater.cc:511] Master 127.31.62.190:37189 requested a full tablet report, sending...
I20260925 10:57:28.452822 12439 ts_manager.cc:194] Registered new tserver with Master: 64c4d0eb70d74aaa93c647e86f62cbfb (127.31.62.129:41361)
I20260925 10:57:28.454036 12439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:52179
W20260925 10:57:28.510988 12781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:28.511507 12781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:28.511708 12781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:28.522886 12781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:28.523102 12781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:28.523319 12781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:57:28.538552 12781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.62.148:40229/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:35015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=35921
--tserver_master_addrs=127.31.62.190:37189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:28.540536 12781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:28.542491 12781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Sep 25 10:57:28 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
W20260925 10:57:28.573541 12790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:28.593295 12791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:28.625319 12781 server_base.cc:1034] running on GCE node
W20260925 10:57:28.626186 12793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:28.627131 12781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:28.633500 12781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:28.634831 12781 hybrid_clock.cc:648] HybridClock initialized: now 1790333848634767 us; error 62 us; skew 500 ppm
I20260925 10:57:28.635212 12781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:28 dist-test-slave-nx7s krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:28.638479 12781 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:57:28.646293 12781 webserver.cc:460] Webserver started at http://127.31.62.131:35921/ using document root <none> and password file <none>
I20260925 10:57:28.647059 12781 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:28.647212 12781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 10:57:28 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:57:28.672058 12781 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.007s sys 0.000s
I20260925 10:57:28.675149 12801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:28.676977 12781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:57:28.677172 12781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "6cdb92dbb0ed4f369e72507161627b6e"
format_stamp: "Formatted at 2026-09-25 10:57:25 on dist-test-slave-nx7s"
server_key: "1c6f088d115cb674c1d214bfe30b3c79"
server_key_iv: "c67aebe70c3fa54c28b460d036d86161"
server_key_version: "kuduclusterkey@0"
I20260925 10:57:28.678197 12781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:28.696118 12781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:28.701135 12781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:28.701428 12781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:28.709352 12781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:57:28.712957 12781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:57:28.713063 12781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:28.713197 12781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:57:28.713264 12781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:28.769672 12781 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:35015
I20260925 10:57:28.769713 12913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:35015 every 8 connection(s)
I20260925 10:57:28.771281 12781 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:57:28.778141 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 12781
Sep 25 10:57:28 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:28.803416 12918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:28.779157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60350 (local address 127.31.62.190:37189)
0925 10:57:28.786118 (+ 6961us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:28.786125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:28.786142 (+ 17us) server_negotiation.cc:407] Connection header received
0925 10:57:28.786201 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:28.786206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:28.786316 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:28.786438 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:28.788561 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.789758 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.791696 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.792263 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.792379 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:28.795895 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:28.795901 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:28.795906 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:28.795944 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:28.799276 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:28.800104 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:28.800109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:28.800111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:28.800261 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:28.800657 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:28.800661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:28.800664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:28.801083 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:28.801308 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:28.802487 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0925 10:57:28.803054 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6837,"spinlock_wait_cycles":10240,"thread_start_us":7055,"threads_started":1}
I20260925 10:57:28.807639 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:28.774637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:42651 (local address 127.31.62.190:37189)
0925 10:57:28.785281 (+ 10644us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:28.785288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:28.785317 (+ 29us) server_negotiation.cc:407] Connection header received
0925 10:57:28.786396 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:28.786402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:28.786521 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:28.786631 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:28.788963 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.790138 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.792575 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.793235 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.793330 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:28.799406 (+ 6076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:28.799414 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:28.799419 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:28.799461 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:28.802871 (+ 3410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:28.803920 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:28.803931 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:28.803934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:28.804074 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:28.805192 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:28.805197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:28.805201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:28.805553 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:28.805689 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:28.807003 (+ 1314us) server_negotiation.cc:299] Negotiation successful
0925 10:57:28.807268 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10502,"spinlock_wait_cycles":7040,"thread_start_us":11022,"threads_started":1}
I20260925 10:57:28.810024 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:28.775252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:37189 (local address 127.31.62.131:42651)
0925 10:57:28.776032 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:28.776081 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:28.785645 (+ 9564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:28.787166 (+ 1521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:28.787181 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:28.787824 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:28.788701 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:28.788730 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.790331 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.790336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:28.792406 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:28.792421 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.793687 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.793695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:28.793993 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:28.794748 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:28.794784 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:28.799068 (+ 4284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:28.803023 (+ 3955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:28.803034 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:28.803055 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:28.803769 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:28.804241 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:28.804246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:28.804250 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:28.805036 (+ 786us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:28.805786 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:28.805795 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:28.806306 (+ 511us) client_negotiation.cc:769] Sending connection context
0925 10:57:28.807853 (+ 1547us) client_negotiation.cc:240] Negotiation successful
0925 10:57:28.808488 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":307,"threads_started":1}
I20260925 10:57:28.811870 12914 heartbeater.cc:347] Connected to a master server at 127.31.62.190:37189
I20260925 10:57:28.812206 12914 heartbeater.cc:464] Registering TS with master...
I20260925 10:57:28.813170 12914 heartbeater.cc:511] Master 127.31.62.190:37189 requested a full tablet report, sending...
I20260925 10:57:28.814913 12439 ts_manager.cc:194] Registered new tserver with Master: 6cdb92dbb0ed4f369e72507161627b6e (127.31.62.131:35015)
I20260925 10:57:28.816033 12439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:42651
I20260925 10:57:28.821309 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:57:28.848268 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:28.825125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60362 (local address 127.31.62.190:37189)
0925 10:57:28.825311 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:28.825317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:28.828806 (+ 3489us) server_negotiation.cc:407] Connection header received
0925 10:57:28.829148 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:28.829155 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:28.829238 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:28.829309 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:28.831455 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.832574 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.834475 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.835061 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.835153 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:28.838552 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:28.838559 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:28.838565 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:28.838614 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:28.842176 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:28.843165 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:28.843171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:28.843175 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:28.843357 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:28.843805 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:28.843809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:28.843813 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:28.844203 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:28.844452 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:28.847491 (+ 3039us) server_negotiation.cc:299] Negotiation successful
0925 10:57:28.847897 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 10:57:28.855154 12439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:57:28.867082 12439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Sep 25 10:57:28 dist-test-slave-nx7s krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333848, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for HTTP/127.31.62.148@KRBTEST.COM
I20260925 10:57:28.983887 12930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:28.914809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:41361 (local address 127.0.0.1:49576)
0925 10:57:28.921450 (+ 6641us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:28.921498 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:28.921678 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:28.964036 (+ 42358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:28.964041 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:28.964068 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:57:28.964458 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:28.964477 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.974265 (+ 9788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.974271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:28.976284 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:28.976301 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.983022 (+ 6721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.983030 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:28.983212 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:28.983253 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 10:57:28.983321 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 10:57:28.983483 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6359,"mutex_wait_us":27,"thread_start_us":6251,"threads_started":1}
I20260925 10:57:28.984754 12932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:28.923299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:44903 (local address 127.0.0.1:57660)
0925 10:57:28.956487 (+ 33188us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:28.956517 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:28.956639 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:28.957803 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:28.957811 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:28.957856 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:57:28.958228 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:28.958241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.962172 (+ 3931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.962177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:28.969477 (+ 7300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:28.969494 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.975513 (+ 6019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.975520 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:28.975777 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:28.975844 (+ 67us) client_negotiation.cc:769] Sending connection context
0925 10:57:28.976003 (+ 159us) client_negotiation.cc:240] Negotiation successful
0925 10:57:28.976255 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33076,"mutex_wait_us":2037,"thread_start_us":32404,"threads_started":1}
I20260925 10:57:28.985308 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:28.915071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49576 (local address 127.31.62.129:41361)
0925 10:57:28.963412 (+ 48341us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:28.963421 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:28.963451 (+ 30us) server_negotiation.cc:407] Connection header received
0925 10:57:28.963545 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:28.963553 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:28.963817 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:28.970811 (+ 6994us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:57:28.971090 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.974123 (+ 3033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.981245 (+ 7122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.982735 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.984429 (+ 1694us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:28.984717 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:28.984806 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 10:57:28.985010 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48139,"thread_start_us":53810,"threads_started":1}
I20260925 10:57:28.985754 12935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:28.923818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57660 (local address 127.31.62.130:44903)
0925 10:57:28.955670 (+ 31852us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:28.955681 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:28.956757 (+ 1076us) server_negotiation.cc:407] Connection header received
0925 10:57:28.956901 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:28.956911 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:28.957222 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:28.957436 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:57:28.958437 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.960033 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.969688 (+ 9655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.974967 (+ 5279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.975108 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:28.975339 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:28.982357 (+ 7018us) server_negotiation.cc:299] Negotiation successful
0925 10:57:28.982585 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31676,"thread_start_us":32111,"threads_started":1}
I20260925 10:57:28.989655 12715 tablet_service.cc:1467] Processing CreateTablet for tablet 129881d46eb24d4e9d4628fad510f128 (DEFAULT_TABLE table=test-table [id=94f1248ddde04e2c9cdfdbae8ee182d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:57:28.989660 12570 tablet_service.cc:1467] Processing CreateTablet for tablet 129881d46eb24d4e9d4628fad510f128 (DEFAULT_TABLE table=test-table [id=94f1248ddde04e2c9cdfdbae8ee182d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:57:28.991269 12570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 129881d46eb24d4e9d4628fad510f128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:57:28.991642 12715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 129881d46eb24d4e9d4628fad510f128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:57:29.005834 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:28.918309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:35015 (local address 127.0.0.1:34968)
0925 10:57:28.955000 (+ 36691us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:28.955026 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:28.955195 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:28.961655 (+ 6460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:28.961659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:28.961687 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:57:28.962016 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:28.962029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.978097 (+ 16068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.978103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:28.980087 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:28.980112 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.004900 (+ 24788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.004909 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:29.005119 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:29.005156 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 10:57:29.005277 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 10:57:29.005450 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36535,"thread_start_us":48719,"threads_started":1}
I20260925 10:57:29.008213 12936 tablet_bootstrap.cc:492] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8: Bootstrap starting.
I20260925 10:57:29.011979 12936 tablet_bootstrap.cc:654] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8: Neither blocks nor log segments found. Creating new log.
I20260925 10:57:29.013331 12936 log.cc:824] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:29.014245 12933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:28.918986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34968 (local address 127.31.62.131:35015)
0925 10:57:28.960637 (+ 41651us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:28.960646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:28.960680 (+ 34us) server_negotiation.cc:407] Connection header received
0925 10:57:28.960777 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:28.960785 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:28.961135 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:28.961341 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:57:28.964355 (+ 3014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.966179 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:28.980398 (+ 14219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:28.982099 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.004802 (+ 22703us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:29.005099 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:29.013647 (+ 8548us) server_negotiation.cc:299] Negotiation successful
0925 10:57:29.013876 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41432,"thread_start_us":41289,"threads_started":1}
I20260925 10:57:29.016136 12937 tablet_bootstrap.cc:492] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb: Bootstrap starting.
I20260925 10:57:29.018487 12849 tablet_service.cc:1467] Processing CreateTablet for tablet 129881d46eb24d4e9d4628fad510f128 (DEFAULT_TABLE table=test-table [id=94f1248ddde04e2c9cdfdbae8ee182d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:57:29.020401 12937 tablet_bootstrap.cc:654] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb: Neither blocks nor log segments found. Creating new log.
I20260925 10:57:29.021728 12937 log.cc:824] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:29.021853 12849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 129881d46eb24d4e9d4628fad510f128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:57:29.037277 12937 tablet_bootstrap.cc:492] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb: No bootstrap required, opened a new log
I20260925 10:57:29.037685 12937 ts_tablet_manager.cc:1397] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.010s
I20260925 10:57:29.047086 12936 tablet_bootstrap.cc:492] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8: No bootstrap required, opened a new log
I20260925 10:57:29.047552 12936 ts_tablet_manager.cc:1397] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8: Time spent bootstrapping tablet: real 0.040s user 0.007s sys 0.004s
I20260925 10:57:29.050537 12937 raft_consensus.cc:348] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c4d0eb70d74aaa93c647e86f62cbfb" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 41361 } } peers { permanent_uuid: "6cdb92dbb0ed4f369e72507161627b6e" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35015 } } peers { permanent_uuid: "661bb149641346a581d2226e9dbc43e8" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44903 } }
I20260925 10:57:29.051172 12937 raft_consensus.cc:374] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:57:29.052158 12937 raft_consensus.cc:729] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64c4d0eb70d74aaa93c647e86f62cbfb, State: Initialized, Role: FOLLOWER
I20260925 10:57:29.053025 12937 consensus_queue.cc:260] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c4d0eb70d74aaa93c647e86f62cbfb" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 41361 } } peers { permanent_uuid: "6cdb92dbb0ed4f369e72507161627b6e" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35015 } } peers { permanent_uuid: "661bb149641346a581d2226e9dbc43e8" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44903 } }
I20260925 10:57:29.053507 12940 tablet_bootstrap.cc:492] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e: Bootstrap starting.
I20260925 10:57:29.059909 12940 tablet_bootstrap.cc:654] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e: Neither blocks nor log segments found. Creating new log.
I20260925 10:57:29.067838 12940 log.cc:824] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:29.070858 12636 heartbeater.cc:503] Master 127.31.62.190:37189 was elected leader, sending a full tablet report...
I20260925 10:57:29.071810 12937 ts_tablet_manager.cc:1428] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb: Time spent starting tablet: real 0.034s user 0.014s sys 0.003s
I20260925 10:57:29.071681 12936 raft_consensus.cc:348] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c4d0eb70d74aaa93c647e86f62cbfb" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 41361 } } peers { permanent_uuid: "6cdb92dbb0ed4f369e72507161627b6e" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35015 } } peers { permanent_uuid: "661bb149641346a581d2226e9dbc43e8" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44903 } }
I20260925 10:57:29.072242 12936 raft_consensus.cc:374] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:57:29.072458 12936 raft_consensus.cc:729] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 661bb149641346a581d2226e9dbc43e8, State: Initialized, Role: FOLLOWER
I20260925 10:57:29.078228 12940 tablet_bootstrap.cc:492] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e: No bootstrap required, opened a new log
I20260925 10:57:29.078527 12940 ts_tablet_manager.cc:1397] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e: Time spent bootstrapping tablet: real 0.025s user 0.011s sys 0.000s
I20260925 10:57:29.078675 12936 consensus_queue.cc:260] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c4d0eb70d74aaa93c647e86f62cbfb" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 41361 } } peers { permanent_uuid: "6cdb92dbb0ed4f369e72507161627b6e" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35015 } } peers { permanent_uuid: "661bb149641346a581d2226e9dbc43e8" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44903 } }
I20260925 10:57:29.087045 12936 ts_tablet_manager.cc:1428] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8: Time spent starting tablet: real 0.039s user 0.011s sys 0.007s
I20260925 10:57:29.087517 12780 heartbeater.cc:503] Master 127.31.62.190:37189 was elected leader, sending a full tablet report...
I20260925 10:57:29.089389 12940 raft_consensus.cc:348] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c4d0eb70d74aaa93c647e86f62cbfb" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 41361 } } peers { permanent_uuid: "6cdb92dbb0ed4f369e72507161627b6e" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35015 } } peers { permanent_uuid: "661bb149641346a581d2226e9dbc43e8" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44903 } }
I20260925 10:57:29.089949 12940 raft_consensus.cc:374] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:57:29.090098 12940 raft_consensus.cc:729] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cdb92dbb0ed4f369e72507161627b6e, State: Initialized, Role: FOLLOWER
I20260925 10:57:29.091060 12940 consensus_queue.cc:260] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c4d0eb70d74aaa93c647e86f62cbfb" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 41361 } } peers { permanent_uuid: "6cdb92dbb0ed4f369e72507161627b6e" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35015 } } peers { permanent_uuid: "661bb149641346a581d2226e9dbc43e8" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44903 } }
I20260925 10:57:29.094900 12914 heartbeater.cc:503] Master 127.31.62.190:37189 was elected leader, sending a full tablet report...
I20260925 10:57:29.095714 12940 ts_tablet_manager.cc:1428] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e: Time spent starting tablet: real 0.017s user 0.018s sys 0.000s
I20260925 10:57:29.141531 12941 raft_consensus.cc:482] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:57:29.141889 12941 raft_consensus.cc:504] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c4d0eb70d74aaa93c647e86f62cbfb" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 41361 } } peers { permanent_uuid: "6cdb92dbb0ed4f369e72507161627b6e" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35015 } } peers { permanent_uuid: "661bb149641346a581d2226e9dbc43e8" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44903 } }
I20260925 10:57:29.144047 12941 leader_election.cc:288] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cdb92dbb0ed4f369e72507161627b6e (127.31.62.131:35015), 661bb149641346a581d2226e9dbc43e8 (127.31.62.130:44903)
W20260925 10:57:29.152199 12637 tablet.cc:2369] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:57:29.156646 12933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:29.146043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:43833 (local address 127.31.62.131:35015)
0925 10:57:29.146566 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:29.146572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:29.149240 (+ 2668us) server_negotiation.cc:407] Connection header received
0925 10:57:29.149599 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:29.149605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:29.149684 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:29.149751 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:57:29.150521 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.151645 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.154261 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.155455 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.155589 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:29.155665 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:29.156230 (+ 565us) server_negotiation.cc:299] Negotiation successful
0925 10:57:29.156372 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374}
I20260925 10:57:29.161620 12935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:29.146344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:40477 (local address 127.31.62.130:44903)
0925 10:57:29.146721 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:29.146727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:29.149414 (+ 2687us) server_negotiation.cc:407] Connection header received
0925 10:57:29.154545 (+ 5131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:29.154550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:29.154662 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:29.154751 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:57:29.155471 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.156581 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.159117 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.160398 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.160526 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:29.160612 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:29.161197 (+ 585us) server_negotiation.cc:299] Negotiation successful
0925 10:57:29.161340 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252}
I20260925 10:57:29.162379 12946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:29.145414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:44903 (local address 127.31.62.129:40477)
0925 10:57:29.149311 (+ 3897us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:29.149330 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:29.154375 (+ 5045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:29.154903 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:29.154908 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:29.154934 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:57:29.155251 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:29.155264 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.156786 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.156792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:29.158893 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:29.158912 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.160815 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.160821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:29.161000 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:29.161034 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 10:57:29.161877 (+ 843us) client_negotiation.cc:240] Negotiation successful
0925 10:57:29.162029 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3775,"thread_start_us":3663,"threads_started":1}
I20260925 10:57:29.163412 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:29.144753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:35015 (local address 127.31.62.129:43833)
0925 10:57:29.149114 (+ 4361us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:29.149137 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:29.149373 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:29.149958 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:29.149963 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:29.149991 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:57:29.150337 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:29.150352 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.151825 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.151830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:29.154071 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:29.154088 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.155853 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.155860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:29.156038 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:29.156074 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 10:57:29.156931 (+ 857us) client_negotiation.cc:240] Negotiation successful
0925 10:57:29.162997 (+ 6066us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4197,"mutex_wait_us":5878,"thread_start_us":3993,"threads_started":1}
I20260925 10:57:29.163473 12735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129881d46eb24d4e9d4628fad510f128" candidate_uuid: "64c4d0eb70d74aaa93c647e86f62cbfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661bb149641346a581d2226e9dbc43e8" is_pre_election: true
I20260925 10:57:29.164115 12735 raft_consensus.cc:2454] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64c4d0eb70d74aaa93c647e86f62cbfb in term 0.
I20260925 10:57:29.164469 12869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129881d46eb24d4e9d4628fad510f128" candidate_uuid: "64c4d0eb70d74aaa93c647e86f62cbfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cdb92dbb0ed4f369e72507161627b6e" is_pre_election: true
I20260925 10:57:29.164996 12524 leader_election.cc:302] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64c4d0eb70d74aaa93c647e86f62cbfb, 661bb149641346a581d2226e9dbc43e8; no voters:
I20260925 10:57:29.165067 12869 raft_consensus.cc:2454] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64c4d0eb70d74aaa93c647e86f62cbfb in term 0.
I20260925 10:57:29.165633 12941 raft_consensus.cc:2781] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:57:29.165786 12941 raft_consensus.cc:482] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:57:29.165901 12941 raft_consensus.cc:3037] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:57:29.168488 12941 raft_consensus.cc:504] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c4d0eb70d74aaa93c647e86f62cbfb" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 41361 } } peers { permanent_uuid: "6cdb92dbb0ed4f369e72507161627b6e" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35015 } } peers { permanent_uuid: "661bb149641346a581d2226e9dbc43e8" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44903 } }
I20260925 10:57:29.169164 12941 leader_election.cc:288] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [CANDIDATE]: Term 1 election: Requested vote from peers 6cdb92dbb0ed4f369e72507161627b6e (127.31.62.131:35015), 661bb149641346a581d2226e9dbc43e8 (127.31.62.130:44903)
I20260925 10:57:29.169557 12869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129881d46eb24d4e9d4628fad510f128" candidate_uuid: "64c4d0eb70d74aaa93c647e86f62cbfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cdb92dbb0ed4f369e72507161627b6e"
I20260925 10:57:29.169775 12869 raft_consensus.cc:3037] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:57:29.169785 12735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129881d46eb24d4e9d4628fad510f128" candidate_uuid: "64c4d0eb70d74aaa93c647e86f62cbfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661bb149641346a581d2226e9dbc43e8"
I20260925 10:57:29.169986 12735 raft_consensus.cc:3037] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:57:29.173041 12735 raft_consensus.cc:2454] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64c4d0eb70d74aaa93c647e86f62cbfb in term 1.
I20260925 10:57:29.173041 12869 raft_consensus.cc:2454] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64c4d0eb70d74aaa93c647e86f62cbfb in term 1.
I20260925 10:57:29.173564 12525 leader_election.cc:302] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64c4d0eb70d74aaa93c647e86f62cbfb, 6cdb92dbb0ed4f369e72507161627b6e; no voters:
I20260925 10:57:29.173934 12941 raft_consensus.cc:2781] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:57:29.174451 12941 raft_consensus.cc:686] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [term 1 LEADER]: Becoming Leader. State: Replica: 64c4d0eb70d74aaa93c647e86f62cbfb, State: Running, Role: LEADER
I20260925 10:57:29.175004 12941 consensus_queue.cc:237] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c4d0eb70d74aaa93c647e86f62cbfb" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 41361 } } peers { permanent_uuid: "6cdb92dbb0ed4f369e72507161627b6e" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35015 } } peers { permanent_uuid: "661bb149641346a581d2226e9dbc43e8" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44903 } }
I20260925 10:57:29.180260 12438 catalog_manager.cc:5495] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 64c4d0eb70d74aaa93c647e86f62cbfb (127.31.62.129). New cstate: current_term: 1 leader_uuid: "64c4d0eb70d74aaa93c647e86f62cbfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64c4d0eb70d74aaa93c647e86f62cbfb" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 41361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cdb92dbb0ed4f369e72507161627b6e" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "661bb149641346a581d2226e9dbc43e8" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44903 } health_report { overall_health: UNKNOWN } } }
I20260925 10:57:29.220485 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:29.214271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49582 (local address 127.31.62.129:41361)
0925 10:57:29.214481 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:29.214494 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:29.214581 (+ 87us) server_negotiation.cc:407] Connection header received
0925 10:57:29.214777 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:29.214783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:29.214905 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:29.215064 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:57:29.215358 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.216513 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.217914 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.218551 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.218678 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:29.219186 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:57:29.219879 (+ 693us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:57:29.220034 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:29.220081 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 10:57:29.220166 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 10:57:29.230322 12869 raft_consensus.cc:1261] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e [term 1 FOLLOWER]: Refusing update from remote peer 64c4d0eb70d74aaa93c647e86f62cbfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:57:29.230561 12735 raft_consensus.cc:1261] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8 [term 1 FOLLOWER]: Refusing update from remote peer 64c4d0eb70d74aaa93c647e86f62cbfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:57:29.231222 12941 consensus_queue.cc:1035] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cdb92dbb0ed4f369e72507161627b6e" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:57:29.231523 12954 consensus_queue.cc:1035] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "661bb149641346a581d2226e9dbc43e8" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 44903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:57:29.251078 12957 mvcc.cc:205] Tried to move back new op lower bound from 7333207446436597760 to 7333207446225514496. Current Snapshot: MvccSnapshot[applied={T|T < 7333207446436597760}]
I20260925 10:57:29.272996 12439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:57:29.273276 12439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:57:29.277086 12439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 385619c6035d47d1b44ebcd34f164fa4: Sending DeleteTablet for 3 replicas of tablet 129881d46eb24d4e9d4628fad510f128
I20260925 10:57:29.278280 12570 tablet_service.cc:1514] Processing DeleteTablet for tablet 129881d46eb24d4e9d4628fad510f128 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:57:29 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:49576
I20260925 10:57:29.278551 12715 tablet_service.cc:1514] Processing DeleteTablet for tablet 129881d46eb24d4e9d4628fad510f128 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:57:29 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:57660
I20260925 10:57:29.279039 12849 tablet_service.cc:1514] Processing DeleteTablet for tablet 129881d46eb24d4e9d4628fad510f128 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:57:29 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:34968
I20260925 10:57:29.287665 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 12476
I20260925 10:57:29.295935 12967 tablet_replica.cc:330] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb: stopping tablet replica
I20260925 10:57:29.296594 12967 raft_consensus.cc:2229] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:57:29.297523 12967 raft_consensus.cc:2258] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:57:29.299034 12968 tablet_replica.cc:330] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8: stopping tablet replica
I20260925 10:57:29.299245 12969 tablet_replica.cc:330] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e: stopping tablet replica
I20260925 10:57:29.299482 12968 raft_consensus.cc:2229] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:57:29.299615 12969 raft_consensus.cc:2229] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:57:29.300009 12968 raft_consensus.cc:2258] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:57:29.300009 12969 raft_consensus.cc:2258] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:57:29.300274 12967 ts_tablet_manager.cc:1905] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:57:29.306192 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:29.289001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49598 (local address 127.31.62.129:41361)
0925 10:57:29.289185 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:29.289192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:29.289521 (+ 329us) server_negotiation.cc:407] Connection header received
0925 10:57:29.289592 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:29.289597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:29.289674 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:29.289760 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:29.291205 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.292501 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.293790 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.294573 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.294696 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:29.297935 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:29.297947 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:29.297963 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:29.298009 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:29.301841 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:29.302795 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:29.302803 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:29.302806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:29.302986 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:29.303436 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:29.303441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:29.303445 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:29.303908 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:29.305355 (+ 1447us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:29.305536 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 10:57:29.305815 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 10:57:29.307019 12631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:29.309134 12969 ts_tablet_manager.cc:1905] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:57:29.315843 12969 ts_tablet_manager.cc:1918] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:57:29.316027 12969 log.cc:1196] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/129881d46eb24d4e9d4628fad510f128
I20260925 10:57:29.316506 12969 ts_tablet_manager.cc:1939] T 129881d46eb24d4e9d4628fad510f128 P 6cdb92dbb0ed4f369e72507161627b6e: Deleting consensus metadata
I20260925 10:57:29.318643 12425 catalog_manager.cc:4841] TS 6cdb92dbb0ed4f369e72507161627b6e (127.31.62.131:35015): tablet 129881d46eb24d4e9d4628fad510f128 (table test-table [id=94f1248ddde04e2c9cdfdbae8ee182d6]) successfully deleted
I20260925 10:57:29.319738 12968 ts_tablet_manager.cc:1905] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:57:29.326385 12968 ts_tablet_manager.cc:1918] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:57:29.326581 12968 log.cc:1196] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/129881d46eb24d4e9d4628fad510f128
I20260925 10:57:29.327042 12968 ts_tablet_manager.cc:1939] T 129881d46eb24d4e9d4628fad510f128 P 661bb149641346a581d2226e9dbc43e8: Deleting consensus metadata
I20260925 10:57:29.328996 12424 catalog_manager.cc:4841] TS 661bb149641346a581d2226e9dbc43e8 (127.31.62.130:44903): tablet 129881d46eb24d4e9d4628fad510f128 (table test-table [id=94f1248ddde04e2c9cdfdbae8ee182d6]) successfully deleted
I20260925 10:57:29.408910 12967 ts_tablet_manager.cc:1918] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:57:29.409186 12967 log.cc:1196] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/129881d46eb24d4e9d4628fad510f128
I20260925 10:57:29.409775 12967 ts_tablet_manager.cc:1939] T 129881d46eb24d4e9d4628fad510f128 P 64c4d0eb70d74aaa93c647e86f62cbfb: Deleting consensus metadata
I20260925 10:57:29.411666 12423 catalog_manager.cc:4841] TS 64c4d0eb70d74aaa93c647e86f62cbfb (127.31.62.129:41361): tablet 129881d46eb24d4e9d4628fad510f128 (table test-table [id=94f1248ddde04e2c9cdfdbae8ee182d6]) successfully deleted
I20260925 10:57:29.417491 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 12476
I20260925 10:57:29.432530 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 12640
I20260925 10:57:29.446241 12935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:29.433359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57672 (local address 127.31.62.130:44903)
0925 10:57:29.433560 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:29.433565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:29.433584 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:57:29.433729 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:29.433734 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:29.433804 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:29.433885 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:29.435197 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.435946 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.437478 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.437881 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.437994 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:29.440474 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:29.440489 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:29.440504 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:29.440542 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:29.443328 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:29.444025 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:29.444034 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:29.444037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:29.444188 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:29.444553 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:29.444559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:29.444561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:29.445042 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:29.445347 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:29.445657 (+ 310us) server_negotiation.cc:299] Negotiation successful
0925 10:57:29.445907 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260925 10:57:29.446959 12775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:29.534291 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 12640
I20260925 10:57:29.547859 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 12781
I20260925 10:57:29.561887 12933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:29.548566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34970 (local address 127.31.62.131:35015)
0925 10:57:29.548762 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:29.548768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:29.548879 (+ 111us) server_negotiation.cc:407] Connection header received
0925 10:57:29.549140 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:29.549145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:29.549206 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:29.549289 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:29.550537 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.551533 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.552835 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:29.553261 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:29.553373 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:29.555842 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:29.555858 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:29.555874 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:29.555999 (+ 125us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:29.558946 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:29.559668 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:29.559675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:29.559677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:29.559802 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:29.560164 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:29.560168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:29.560170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:29.560521 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:29.560792 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:29.561357 (+ 565us) server_negotiation.cc:299] Negotiation successful
0925 10:57:29.561576 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 10:57:29.562497 12909 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:29.650544 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 12781
I20260925 10:57:29.664026 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 12406
I20260925 10:57:29.665243 12469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:29.777367 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 12406
I20260925 10:57:29.794791 31994 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 10:57:30.129767 31994 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 10:57:30.129974 31994 mini_ranger.cc:66] Stopping Ranger...
I20260925 10:57:30.465554 31994 mini_ranger.cc:68] Stopped Ranger
2026-09-25 10:57:30.465 UTC [11459] LOG: received smart shutdown request
2026-09-25 10:57:30.468 UTC [11459] LOG: background worker "logical replication launcher" (PID 11468) exited with exit code 1
2026-09-25 10:57:30.468 UTC [11463] LOG: shutting down
2026-09-25 10:57:30.645 UTC [11459] LOG: database system is shut down
2026-09-25T10:57:30Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53013 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:57:30 dist-test-slave-nx7s krb5kdc[12982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:57:30 dist-test-slave-nx7s krb5kdc[12982](info): set up 2 sockets
Sep 25 10:57:30 dist-test-slave-nx7s krb5kdc[12982](info): commencing operation
krb5kdc: starting...
W20260925 10:57:31.765523 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:57:31 dist-test-slave-nx7s krb5kdc[12982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:57:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:57:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:57:31.986411 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:44061
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:36709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:44061,127.31.62.189:35409,127.31.62.188:40939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:57:32.312282 12995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:32.312615 12995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:32.312753 12995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:32.320046 12995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:57:32.320178 12995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:32.320263 12995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:32.320338 12995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:57:32.320398 12995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:57:32.330372 12995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:36709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:44061,127.31.62.189:35409,127.31.62.188:40939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:44061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:32.331779 12995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:32.333174 12995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:32.344772 13000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:32.344709 13001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:32.344743 13003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:32.345578 12995 server_base.cc:1034] running on GCE node
I20260925 10:57:32.346359 12995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:32.347821 12995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:32.349000 12995 hybrid_clock.cc:648] HybridClock initialized: now 1790333852348954 us; error 41 us; skew 500 ppm
I20260925 10:57:32.349349 12995 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:32 dist-test-slave-nx7s krb5kdc[12982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333852, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:32.352360 12995 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:57:32.353920 12995 webserver.cc:460] Webserver started at http://127.31.62.190:38537/ using document root <none> and password file <none>
I20260925 10:57:32.354598 12995 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:32.354697 12995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:32.354964 12995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:57:32.358106 12995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "6db3cb9a53f64268adcca95e2c1fce3d"
format_stamp: "Formatted at 2026-09-25 10:57:32 on dist-test-slave-nx7s"
server_key: "d8dff6c7b046e2296780f78f781d8348"
server_key_iv: "d2c118e23d395f836a31dbf140f931e5"
server_key_version: "encryptionkey@0"
I20260925 10:57:32.358927 12995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "6db3cb9a53f64268adcca95e2c1fce3d"
format_stamp: "Formatted at 2026-09-25 10:57:32 on dist-test-slave-nx7s"
server_key: "d8dff6c7b046e2296780f78f781d8348"
server_key_iv: "d2c118e23d395f836a31dbf140f931e5"
server_key_version: "encryptionkey@0"
I20260925 10:57:32.364267 12995 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260925 10:57:32.367780 13010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:32.369190 12995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260925 10:57:32.369347 12995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "6db3cb9a53f64268adcca95e2c1fce3d"
format_stamp: "Formatted at 2026-09-25 10:57:32 on dist-test-slave-nx7s"
server_key: "d8dff6c7b046e2296780f78f781d8348"
server_key_iv: "d2c118e23d395f836a31dbf140f931e5"
server_key_version: "encryptionkey@0"
I20260925 10:57:32.369527 12995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:32.388401 12995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:32.392654 12995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:32.392961 12995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:32.410281 12995 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:44061
I20260925 10:57:32.410305 13061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:44061 every 8 connection(s)
I20260925 10:57:32.411681 12995 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:57:32.415488 13062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:57:32.417130 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 12995
I20260925 10:57:32.417358 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:57:32.418169 31994 external_mini_cluster.cc:1428] Setting key f2f5dced9a6cc8034daadda55237a962
I20260925 10:57:32.424803 13062 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.62.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.189@KRBTEST.COM" created.
Sep 25 10:57:32 dist-test-slave-nx7s krb5kdc[12982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333852, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:32.450186 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.428493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44061 (local address 127.0.0.1:54698)
0925 10:57:32.429354 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:32.429410 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:32.431535 (+ 2125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:32.432555 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:32.432560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:32.433200 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:32.433553 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:32.433572 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:32.435710 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:32.435714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:32.436840 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:32.436886 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:32.438170 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:32.438176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:32.438317 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:32.439011 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:32.439034 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:32.443071 (+ 4037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:32.446192 (+ 3121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:32.446207 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:32.446219 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:32.446751 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:32.447135 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:32.447139 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:32.447142 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:32.447352 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:32.448118 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:32.448126 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:32.448380 (+ 254us) client_negotiation.cc:769] Sending connection context
0925 10:57:32.448612 (+ 232us) client_negotiation.cc:240] Negotiation successful
0925 10:57:32.449080 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":353,"threads_started":1}
I20260925 10:57:32.450210 13066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.429883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54698 (local address 127.31.62.190:44061)
0925 10:57:32.430582 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:32.430591 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:32.430648 (+ 57us) server_negotiation.cc:407] Connection header received
0925 10:57:32.431948 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:32.431955 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:32.432302 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:32.432531 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:32.433755 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:32.435539 (+ 1784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:32.437029 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:32.437921 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:32.438140 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:32.443343 (+ 5203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:32.443352 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:32.443365 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:32.443399 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:32.446035 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:32.446878 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:32.446885 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:32.446887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:32.447037 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:32.447450 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:32.447453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:32.447455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:32.447981 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:32.448155 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:32.448830 (+ 675us) server_negotiation.cc:299] Negotiation successful
0925 10:57:32.449258 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":490,"thread_start_us":320,"threads_started":1}
I20260925 10:57:32.453446 13062 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } has no permanent_uuid. Determining permanent_uuid...
I20260925 10:57:32.454638 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.454218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:35409 (local address 127.0.0.1:51282)
0925 10:57:32.454390 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:32.454497 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260925 10:57:32.454949 13012 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
I20260925 10:57:32.456439 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.456085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:35409 (local address 127.0.0.1:51286)
0925 10:57:32.456236 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:32.456316 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 10:57:32.456629 13062 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.189:35409: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.189.keytab.
Entry for principal kudu/127.31.62.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.189.keytab.
I20260925 10:57:32.471884 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.189:35409
--webserver_interface=127.31.62.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.189.keytab
--principal=kudu/127.31.62.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:36709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:44061,127.31.62.189:35409,127.31.62.188:40939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:57:32.505954 13062 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } attempt: 1
I20260925 10:57:32.507184 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.506663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:35409 (local address 127.0.0.1:51300)
0925 10:57:32.506907 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:32.507041 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260925 10:57:32.508281 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.507850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:35409 (local address 127.0.0.1:51302)
0925 10:57:32.508072 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:32.508146 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260925 10:57:32.508452 13062 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.189:35409: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
I20260925 10:57:32.576712 13062 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } attempt: 2
I20260925 10:57:32.577854 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.577369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:35409 (local address 127.0.0.1:51314)
0925 10:57:32.577619 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:32.577716 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260925 10:57:32.578902 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.578530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:35409 (local address 127.0.0.1:51330)
0925 10:57:32.578693 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:32.578776 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260925 10:57:32.579070 13062 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.189:35409: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
I20260925 10:57:32.670296 13062 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } attempt: 3
I20260925 10:57:32.671348 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.670932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:35409 (local address 127.0.0.1:51338)
0925 10:57:32.671125 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:32.671215 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260925 10:57:32.672428 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.672049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:35409 (local address 127.0.0.1:51354)
0925 10:57:32.672201 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:32.672289 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260925 10:57:32.672641 13062 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.189:35409: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
W20260925 10:57:32.793387 13069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:32.793721 13069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:32.793853 13069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:32.801227 13069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:57:32.801357 13069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:32.801438 13069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:32.801512 13069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:57:32.801571 13069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:57:32.813580 13069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:36709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:44061,127.31.62.189:35409,127.31.62.188:40939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.189.keytab
--principal=kudu/127.31.62.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.189:35409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.62.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:32.815100 13069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:32.815876 13062 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } attempt: 4
I20260925 10:57:32.816565 13069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 10:57:32.817003 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.816461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:35409 (local address 127.0.0.1:51360)
0925 10:57:32.816718 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:32.816819 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260925 10:57:32.818094 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.817686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:35409 (local address 127.0.0.1:51370)
0925 10:57:32.817837 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:32.817945 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 10:57:32.818279 13062 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.189:35409: Network error: Client connection negotiation failed: client connection to 127.31.62.189:35409: connect: Connection refused (error 111)
W20260925 10:57:32.829222 13074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:32.829324 13077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:32.829185 13075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:32.829836 13069 server_base.cc:1034] running on GCE node
I20260925 10:57:32.830694 13069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:32.832226 13069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:32.833438 13069 hybrid_clock.cc:648] HybridClock initialized: now 1790333852833371 us; error 60 us; skew 500 ppm
I20260925 10:57:32.833817 13069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:32 dist-test-slave-nx7s krb5kdc[12982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333852, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:32.836939 13069 init.cc:377] Logged in from keytab as kudu/127.31.62.189@KRBTEST.COM (short username kudu)
I20260925 10:57:32.838371 13069 webserver.cc:460] Webserver started at http://127.31.62.189:36221/ using document root <none> and password file <none>
I20260925 10:57:32.839098 13069 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:32.839215 13069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:32.839497 13069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:57:32.842763 13069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/data/instance:
uuid: "68800bc00c714ff48f16845ebeeb08bc"
format_stamp: "Formatted at 2026-09-25 10:57:32 on dist-test-slave-nx7s"
server_key: "88c3b8eae7e1dca6a08387a772d29e8a"
server_key_iv: "4fabd16c76945752583a13c804cc958e"
server_key_version: "encryptionkey@0"
I20260925 10:57:32.843595 13069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/wal/instance:
uuid: "68800bc00c714ff48f16845ebeeb08bc"
format_stamp: "Formatted at 2026-09-25 10:57:32 on dist-test-slave-nx7s"
server_key: "88c3b8eae7e1dca6a08387a772d29e8a"
server_key_iv: "4fabd16c76945752583a13c804cc958e"
server_key_version: "encryptionkey@0"
I20260925 10:57:32.849879 13069 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260925 10:57:32.853673 13084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:32.855247 13069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:57:32.855499 13069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/wal
uuid: "68800bc00c714ff48f16845ebeeb08bc"
format_stamp: "Formatted at 2026-09-25 10:57:32 on dist-test-slave-nx7s"
server_key: "88c3b8eae7e1dca6a08387a772d29e8a"
server_key_iv: "4fabd16c76945752583a13c804cc958e"
server_key_version: "encryptionkey@0"
I20260925 10:57:32.855710 13069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:32.873847 13069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:32.878221 13069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:32.878484 13069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:32.895393 13069 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.189:35409
I20260925 10:57:32.895422 13135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.189:35409 every 8 connection(s)
I20260925 10:57:32.896768 13069 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/data/info.pb
I20260925 10:57:32.900489 13136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:57:32.902247 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 13069
I20260925 10:57:32.902439 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-1/wal/instance
I20260925 10:57:32.902868 31994 external_mini_cluster.cc:1428] Setting key a2e992c0cdcbf68c8aa9ad8d58f8b4a0
I20260925 10:57:32.907176 13136 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.62.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.188@KRBTEST.COM" created.
Sep 25 10:57:32 dist-test-slave-nx7s krb5kdc[12982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333852, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.189@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:32.933152 13066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.909748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54704 (local address 127.31.62.190:44061)
0925 10:57:32.909912 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:32.909916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:32.911211 (+ 1295us) server_negotiation.cc:407] Connection header received
0925 10:57:32.913638 (+ 2427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:32.913642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:32.913711 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:32.913795 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:32.915812 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:32.916472 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:32.918674 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:32.919070 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:32.919200 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:32.927068 (+ 7868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:32.927074 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:32.927077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:32.927124 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:32.929626 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:32.930605 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:32.930610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:32.930612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:32.930738 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:32.931190 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:32.931193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:32.931195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:32.931466 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:32.931575 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:32.932533 (+ 958us) server_negotiation.cc:299] Negotiation successful
0925 10:57:32.932780 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 10:57:32.934039 13139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.910204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44061 (local address 127.0.0.1:54704)
0925 10:57:32.910993 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:32.911052 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:32.913356 (+ 2304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:32.914061 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:32.914073 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:32.914828 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:32.915635 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:32.915660 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:32.916626 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:32.916631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:32.918537 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:32.918553 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:32.919317 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:32.919324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:32.919546 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:32.923967 (+ 4421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:32.923993 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:32.926761 (+ 2768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:32.929824 (+ 3063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:32.929835 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:32.929854 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:32.930463 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:32.930833 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:32.930838 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:32.930841 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:32.931098 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:32.931575 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:32.931583 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:32.932109 (+ 526us) client_negotiation.cc:769] Sending connection context
0925 10:57:32.932392 (+ 283us) client_negotiation.cc:240] Negotiation successful
0925 10:57:32.932807 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":300,"threads_started":1}
I20260925 10:57:32.935755 13136 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 10:57:32 dist-test-slave-nx7s krb5kdc[12982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333852, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.189@KRBTEST.COM for kudu/127.31.62.189@KRBTEST.COM
I20260925 10:57:32.951436 13139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.936551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:35409 (local address 127.0.0.1:51372)
0925 10:57:32.936738 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:32.936757 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:32.936930 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:32.938017 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:32.938022 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:32.938504 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:32.938683 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:32.938693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:32.940231 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:32.940236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:32.941560 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:32.941576 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:32.942440 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:32.942446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:32.942647 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:32.943279 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:32.943303 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:32.945715 (+ 2412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:32.949018 (+ 3303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:32.949022 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:32.949024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:32.949477 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:32.949870 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:32.949875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:32.949879 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:32.950014 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:32.950629 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:32.950636 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:32.950753 (+ 117us) client_negotiation.cc:769] Sending connection context
0925 10:57:32.950857 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 10:57:32.951103 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 10:57:32.951547 13141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.936834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51372 (local address 127.31.62.189:35409)
0925 10:57:32.937596 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:32.937602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:32.937626 (+ 24us) server_negotiation.cc:407] Connection header received
0925 10:57:32.937669 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:32.937675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:32.937859 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:32.937979 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:32.938852 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:32.940097 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:32.941708 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:32.942254 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:32.942354 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:32.945920 (+ 3566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:32.945937 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:32.945957 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:32.946051 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:32.948889 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:32.949632 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:32.949640 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:32.949643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:32.949769 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:32.950126 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:32.950129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:32.950131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:32.950515 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:32.950709 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:32.951008 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 10:57:32.951249 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":445,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.188.keytab.
Entry for principal kudu/127.31.62.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.188.keytab.
I20260925 10:57:32.954048 13136 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } has no permanent_uuid. Determining permanent_uuid...
I20260925 10:57:32.954509 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.188:40939
--webserver_interface=127.31.62.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.188.keytab
--principal=kudu/127.31.62.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:36709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:44061,127.31.62.189:35409,127.31.62.188:40939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:57:32.955209 13139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.954762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41754)
0925 10:57:32.954938 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:32.955055 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 10:57:32.955474 13088 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
I20260925 10:57:32.956985 13139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:32.956623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41756)
0925 10:57:32.956758 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:32.956838 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 10:57:32.957226 13136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:40939: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
I20260925 10:57:33.006605 13136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } attempt: 1
I20260925 10:57:33.007814 13139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.007321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41766)
0925 10:57:33.007523 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.007657 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260925 10:57:33.008956 13139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.008539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41772)
0925 10:57:33.008704 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.008786 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260925 10:57:33.009177 13136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:40939: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
I20260925 10:57:33.077373 13136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } attempt: 2
I20260925 10:57:33.078474 13139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.078039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41786)
0925 10:57:33.078229 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.078320 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260925 10:57:33.079510 13139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.079177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41798)
0925 10:57:33.079322 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.079393 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260925 10:57:33.079787 13136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:40939: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
I20260925 10:57:33.117511 13062 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } attempt: 5
Sep 25 10:57:33 dist-test-slave-nx7s krb5kdc[12982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333852, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for kudu/127.31.62.189@KRBTEST.COM
I20260925 10:57:33.134799 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.118287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:35409 (local address 127.0.0.1:51382)
0925 10:57:33.118624 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.118645 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:33.118776 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:33.119168 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:33.119173 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:33.119653 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:33.119896 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:33.119909 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.121598 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.121604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:33.122972 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:33.122990 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.124081 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.124088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:33.124292 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.124975 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:33.125001 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:33.127850 (+ 2849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:33.131845 (+ 3995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:33.131850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:33.131853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:33.132375 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:33.132773 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:33.132776 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:33.132778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:33.132959 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:33.133746 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:33.133752 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:33.133950 (+ 198us) client_negotiation.cc:769] Sending connection context
0925 10:57:33.134118 (+ 168us) client_negotiation.cc:240] Negotiation successful
0925 10:57:33.134417 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199}
I20260925 10:57:33.136112 13141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.118428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51382 (local address 127.31.62.189:35409)
0925 10:57:33.118766 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:33.118772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:33.118794 (+ 22us) server_negotiation.cc:407] Connection header received
0925 10:57:33.118895 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:33.118900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:33.118997 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:33.119106 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:33.120169 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.121278 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.123175 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.123882 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.124027 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.128113 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:33.128121 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:33.128127 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:33.128175 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:33.131677 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.132505 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.132510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.132514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.132667 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.133140 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.133150 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.133153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.133528 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:33.133678 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:33.135547 (+ 1869us) server_negotiation.cc:299] Negotiation successful
0925 10:57:33.135790 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 10:57:33.137398 13062 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } has no permanent_uuid. Determining permanent_uuid...
I20260925 10:57:33.139263 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.138417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41806)
0925 10:57:33.138837 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.138963 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245}
I20260925 10:57:33.141553 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.140424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41820)
0925 10:57:33.141118 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.141292 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":304,"threads_started":1}
W20260925 10:57:33.141824 13062 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:40939: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
I20260925 10:57:33.171027 13136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } attempt: 3
I20260925 10:57:33.172168 13139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.171711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41832)
0925 10:57:33.171908 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.172008 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 10:57:33.173389 13139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.172929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41834)
0925 10:57:33.173181 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.173267 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260925 10:57:33.173585 13136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:40939: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
I20260925 10:57:33.193073 13062 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } attempt: 1
I20260925 10:57:33.194180 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.193758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41836)
0925 10:57:33.193950 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.194038 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260925 10:57:33.195287 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.194914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41852)
0925 10:57:33.195073 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.195162 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260925 10:57:33.195525 13062 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:40939: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
I20260925 10:57:33.263733 13062 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } attempt: 2
I20260925 10:57:33.264971 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.264465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41864)
0925 10:57:33.264685 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.264781 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260925 10:57:33.266012 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.265658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41880)
0925 10:57:33.265796 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.265871 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 10:57:33.266220 13062 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:40939: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
W20260925 10:57:33.289012 13143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:33.289333 13143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:33.289423 13143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:33.296792 13143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:57:33.296962 13143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:33.297022 13143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:33.297078 13143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:57:33.297122 13143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:57:33.307380 13143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:36709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:44061,127.31.62.189:35409,127.31.62.188:40939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.188.keytab
--principal=kudu/127.31.62.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.188:40939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.62.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:33.308691 13143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:33.310024 13143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 10:57:33.316807 13136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } attempt: 4
I20260925 10:57:33.317881 13139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.317391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41896)
0925 10:57:33.317632 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.317729 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260925 10:57:33.318907 13139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.318572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41898)
0925 10:57:33.318720 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.318791 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 10:57:33.319185 13136 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:40939: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
W20260925 10:57:33.321751 13149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:33.322013 13152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:33.322487 13150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:33.322888 13143 server_base.cc:1034] running on GCE node
I20260925 10:57:33.323745 13143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:33.325407 13143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:33.326640 13143 hybrid_clock.cc:648] HybridClock initialized: now 1790333853326578 us; error 52 us; skew 500 ppm
I20260925 10:57:33.327057 13143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:33 dist-test-slave-nx7s krb5kdc[12982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333853, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:33.330170 13143 init.cc:377] Logged in from keytab as kudu/127.31.62.188@KRBTEST.COM (short username kudu)
I20260925 10:57:33.331662 13143 webserver.cc:460] Webserver started at http://127.31.62.188:38905/ using document root <none> and password file <none>
I20260925 10:57:33.332417 13143 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:33.332536 13143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:33.332826 13143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:57:33.336054 13143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/data/instance:
uuid: "b2bee0749cbc4dff94f798be44003929"
format_stamp: "Formatted at 2026-09-25 10:57:33 on dist-test-slave-nx7s"
server_key: "15ec787328661a58e8bb27db070b7f6d"
server_key_iv: "440d83591be7cb62d6479e9846d7b05d"
server_key_version: "encryptionkey@0"
I20260925 10:57:33.336920 13143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/wal/instance:
uuid: "b2bee0749cbc4dff94f798be44003929"
format_stamp: "Formatted at 2026-09-25 10:57:33 on dist-test-slave-nx7s"
server_key: "15ec787328661a58e8bb27db070b7f6d"
server_key_iv: "440d83591be7cb62d6479e9846d7b05d"
server_key_version: "encryptionkey@0"
I20260925 10:57:33.342234 13143 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260925 10:57:33.345717 13159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:33.347072 13143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:57:33.347245 13143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/wal
uuid: "b2bee0749cbc4dff94f798be44003929"
format_stamp: "Formatted at 2026-09-25 10:57:33 on dist-test-slave-nx7s"
server_key: "15ec787328661a58e8bb27db070b7f6d"
server_key_iv: "440d83591be7cb62d6479e9846d7b05d"
server_key_version: "encryptionkey@0"
I20260925 10:57:33.347430 13143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:33.371542 13143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:33.372452 13062 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } attempt: 3
I20260925 10:57:33.373638 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.373128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41910)
0925 10:57:33.373362 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.373466 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260925 10:57:33.374615 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.374234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41922)
0925 10:57:33.374416 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.374487 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260925 10:57:33.374848 13062 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:40939: Network error: Client connection negotiation failed: client connection to 127.31.62.188:40939: connect: Connection refused (error 111)
I20260925 10:57:33.376354 13143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:33.376621 13143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:33.392944 13143 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.188:40939
I20260925 10:57:33.393016 13210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.188:40939 every 8 connection(s)
I20260925 10:57:33.394326 13143 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/data/info.pb
I20260925 10:57:33.397891 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 13143
I20260925 10:57:33.398053 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790333647382755-31994-0/minicluster-data/master-2/wal/instance
I20260925 10:57:33.398275 13211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:57:33.398526 31994 external_mini_cluster.cc:1428] Setting key 3fc65259024c3072c2910df12d215547
I20260925 10:57:33.408340 13211 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 10:57:33 dist-test-slave-nx7s krb5kdc[12982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.188@KRBTEST.COM
Sep 25 10:57:33 dist-test-slave-nx7s krb5kdc[12982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333853, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.188@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:33.423552 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.400399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41928 (local address 127.31.62.188:40939)
0925 10:57:33.401272 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:33.401321 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:33.401381 (+ 60us) server_negotiation.cc:407] Connection header received
0925 10:57:33.403152 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:33.403181 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:33.403727 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:33.404179 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:33.405557 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.407646 (+ 2089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.409136 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.410138 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.410313 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.415266 (+ 4953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:33.415278 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:33.415292 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:33.415345 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:33.419242 (+ 3897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.420148 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.420156 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.420165 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.420391 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.420827 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.420831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.420833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.421364 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:33.421638 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:33.422135 (+ 497us) server_negotiation.cc:299] Negotiation successful
0925 10:57:33.422532 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"spinlock_wait_cycles":19712,"thread_start_us":377,"threads_started":1}
I20260925 10:57:33.425961 13066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.411272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54712 (local address 127.31.62.190:44061)
0925 10:57:33.411490 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:33.411496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:33.412193 (+ 697us) server_negotiation.cc:407] Connection header received
0925 10:57:33.412471 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:33.412477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:33.412546 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:33.412635 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:33.413811 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.414886 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.416458 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.416979 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.417370 (+ 391us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.420420 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:33.420425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:33.420428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:33.420461 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:33.423014 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.423802 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.423807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.423810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.423970 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.424404 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.424409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.424411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.424743 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:33.424948 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:33.425379 (+ 431us) server_negotiation.cc:299] Negotiation successful
0925 10:57:33.425603 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 10:57:33.425997 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.411274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44061 (local address 127.0.0.1:54712)
0925 10:57:33.412054 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.412104 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:33.412313 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:33.412719 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:33.412726 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:33.413372 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:33.413649 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:33.413660 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.415069 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.415075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:33.416306 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:33.416323 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.417176 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.417183 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:33.417383 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.417981 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:33.418001 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:33.420234 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:33.423135 (+ 2901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:33.423146 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:33.423153 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:33.423556 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:33.424103 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:33.424108 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:33.424111 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:33.424290 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:33.424914 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:33.424920 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:33.425060 (+ 140us) client_negotiation.cc:769] Sending connection context
0925 10:57:33.425250 (+ 190us) client_negotiation.cc:240] Negotiation successful
0925 10:57:33.425570 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":412,"threads_started":1}
I20260925 10:57:33.427449 13211 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 10:57:33 dist-test-slave-nx7s krb5kdc[12982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333853, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.188@KRBTEST.COM for kudu/127.31.62.189@KRBTEST.COM
Sep 25 10:57:33 dist-test-slave-nx7s krb5kdc[12982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:33.440287 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.428048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:35409 (local address 127.0.0.1:51396)
0925 10:57:33.428191 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.428207 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:33.428294 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:33.428578 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:33.428582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:33.428962 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:33.429146 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:33.429158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.430108 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.430112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:33.430960 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:33.430970 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.431631 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.431637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:33.431839 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.432381 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:33.432397 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:33.434526 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:33.437777 (+ 3251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:33.437783 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:33.437786 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:33.438325 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:33.438737 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:33.438742 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:33.438745 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:33.438913 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:33.439463 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:33.439467 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:33.439625 (+ 158us) client_negotiation.cc:769] Sending connection context
0925 10:57:33.439738 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 10:57:33.440021 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 10:57:33.440382 13141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.428165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51396 (local address 127.31.62.189:35409)
0925 10:57:33.428328 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:33.428335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:33.428352 (+ 17us) server_negotiation.cc:407] Connection header received
0925 10:57:33.428404 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:33.428409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:33.428468 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:33.428543 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:33.429286 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.429972 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.431089 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.431494 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.431596 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.434736 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:33.434743 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:33.434748 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:33.434795 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:33.437618 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.438451 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.438456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.438458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.438623 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.439006 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.439010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.439013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.439358 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:33.439530 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:33.439850 (+ 320us) server_negotiation.cc:299] Negotiation successful
0925 10:57:33.440018 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 10:57:33.441260 13211 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } has no permanent_uuid. Determining permanent_uuid...
I20260925 10:57:33.441977 13066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.427187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54726 (local address 127.31.62.190:44061)
0925 10:57:33.427331 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:33.427336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:33.427375 (+ 39us) server_negotiation.cc:407] Connection header received
0925 10:57:33.427542 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:33.427547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:33.427617 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:33.427696 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:33.428992 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.429814 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.431095 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.431510 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.431628 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.436035 (+ 4407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:33.436040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:33.436043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:33.436075 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:33.438897 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.439807 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.439813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.439816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.439971 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.440549 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.440554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.440556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.440925 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:33.441192 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:33.441465 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 10:57:33.441683 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Sep 25 10:57:33 dist-test-slave-nx7s krb5kdc[12982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333853, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.188@KRBTEST.COM for kudu/127.31.62.188@KRBTEST.COM
Sep 25 10:57:33 dist-test-slave-nx7s krb5kdc[12982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.189@KRBTEST.COM
I20260925 10:57:33.454245 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.442028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41936)
0925 10:57:33.442288 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.442306 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:33.442410 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:33.442767 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:33.442771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:33.443186 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:33.443354 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:33.443365 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.444382 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.444387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:33.445297 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:33.445309 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.445947 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.445953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:33.446109 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.446622 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:33.446638 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:33.448650 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:33.451834 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:33.451840 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:33.451845 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:33.452326 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:33.452682 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:33.452688 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:33.452691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:33.452813 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:33.453292 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:33.453296 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:33.453431 (+ 135us) client_negotiation.cc:769] Sending connection context
0925 10:57:33.453543 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 10:57:33.453836 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260925 10:57:33.454294 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.442031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41936 (local address 127.31.62.188:40939)
0925 10:57:33.442204 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:33.442209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:33.442371 (+ 162us) server_negotiation.cc:407] Connection header received
0925 10:57:33.442546 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:33.442551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:33.442635 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:33.442732 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:33.443496 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.444254 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.445414 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.445808 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.445908 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.448811 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:33.448815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:33.448818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:33.448878 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:33.451677 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.452454 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.452458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.452461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.452576 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.452917 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.452922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.452925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.453188 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:33.453314 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:33.453644 (+ 330us) server_negotiation.cc:299] Negotiation successful
0925 10:57:33.454004 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":133,"server-negotiator.queue_time_us":42}
I20260925 10:57:33.457510 13141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.443587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.31.62.189:35409)
0925 10:57:33.443734 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:33.443739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:33.443783 (+ 44us) server_negotiation.cc:407] Connection header received
0925 10:57:33.443957 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:33.443963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:33.444025 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:33.444108 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:33.445331 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.446022 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.447238 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.447639 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.447743 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.452219 (+ 4476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:33.452224 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:33.452227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:33.452261 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:33.454922 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.455576 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.455581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.455585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.455737 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.456168 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.456172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.456174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.456444 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:33.456655 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:33.457065 (+ 410us) server_negotiation.cc:299] Negotiation successful
0925 10:57:33.457269 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 10:57:33.460242 13211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929: Bootstrap starting.
I20260925 10:57:33.462661 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:57:33.463732 13211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929: Neither blocks nor log segments found. Creating new log.
I20260925 10:57:33.464910 13211 log.cc:824] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:33.471530 13211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929: No bootstrap required, opened a new log
I20260925 10:57:33.483263 13211 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } }
I20260925 10:57:33.483727 13211 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:57:33.483851 13211 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2bee0749cbc4dff94f798be44003929, State: Initialized, Role: FOLLOWER
I20260925 10:57:33.484565 13211 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } }
I20260925 10:57:33.489320 13224 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } } }
I20260925 10:57:33.489902 13224 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 10:57:33.490688 13211 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:57:33.491852 13066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.470904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54742 (local address 127.31.62.190:44061)
0925 10:57:33.471088 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:33.471106 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:33.471837 (+ 731us) server_negotiation.cc:407] Connection header received
0925 10:57:33.472134 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:33.472139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:33.472221 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:33.472296 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:33.475117 (+ 2821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.476166 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.478507 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.479127 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.479229 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.483678 (+ 4449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:33.483685 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:33.483689 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:33.483735 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:33.487199 (+ 3464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.488366 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.488373 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.488376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.488566 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.489387 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.489392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.489395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.490101 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:33.490360 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:33.491018 (+ 658us) server_negotiation.cc:299] Negotiation successful
0925 10:57:33.491484 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 10:57:33.505754 13141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.471444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51408 (local address 127.31.62.189:35409)
0925 10:57:33.471621 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:33.471627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:33.475670 (+ 4043us) server_negotiation.cc:407] Connection header received
0925 10:57:33.475847 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:33.475853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:33.475936 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:33.475999 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:33.478520 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.479540 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.481678 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.482297 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.482397 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.486775 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:33.486783 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:33.486789 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:33.486838 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:33.501551 (+ 14713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.502635 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.502641 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.502645 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.502808 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.503242 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.503248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.503251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.503650 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:33.504913 (+ 1263us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:33.505061 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 10:57:33.505298 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 10:57:33.511584 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.472030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41938 (local address 127.31.62.188:40939)
0925 10:57:33.472204 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:33.472209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:33.483470 (+ 11261us) server_negotiation.cc:407] Connection header received
0925 10:57:33.483682 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:33.483688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:33.483778 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:33.483855 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:33.485627 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.486693 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.488687 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.499466 (+ 10779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.499612 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.503270 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:33.503276 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:33.503281 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:33.503322 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:33.507136 (+ 3814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.507933 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.507938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.507941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.508092 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.508523 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.508527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.508529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.508942 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:33.510531 (+ 1589us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:33.510704 (+ 173us) server_negotiation.cc:299] Negotiation successful
0925 10:57:33.511004 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 10:57:33.506016 13143 master_runner.cc:186] Error getting master registration for 127.31.62.190:44061: OK, instance_id { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" instance_seqno: 1790333852405385 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 10:57:33.515332 13143 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 10:57:33.515990 13235 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 10:57:33.516127 13235 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 10:57:33.552079 13062 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } attempt: 4
Sep 25 10:57:33 dist-test-slave-nx7s krb5kdc[12982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333852, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for kudu/127.31.62.188@KRBTEST.COM
I20260925 10:57:33.568151 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.552870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41944 (local address 127.31.62.188:40939)
0925 10:57:33.553026 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:33.553030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:33.553125 (+ 95us) server_negotiation.cc:407] Connection header received
0925 10:57:33.553329 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:33.553333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:33.553416 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:33.553495 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:33.554808 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.555856 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.557471 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.557918 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.558040 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.561362 (+ 3322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:33.561370 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:33.561375 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:33.561429 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:33.565007 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.565763 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.565769 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.565773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.565938 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.566278 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.566283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.566287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.566713 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:33.566862 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:33.567539 (+ 677us) server_negotiation.cc:299] Negotiation successful
0925 10:57:33.567774 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 10:57:33.568653 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.552760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41944)
0925 10:57:33.553010 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.553034 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:33.553194 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:33.553555 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:33.553561 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:33.554187 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:33.554662 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:33.554679 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.556058 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.556063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:33.557336 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:33.557350 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.558071 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.558077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:33.558258 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.558845 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:33.558862 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:33.561168 (+ 2306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:33.565145 (+ 3977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:33.565150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:33.565153 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:33.565636 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:33.566033 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:33.566036 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:33.566038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:33.566167 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:33.567030 (+ 863us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:33.567034 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:33.567238 (+ 204us) client_negotiation.cc:769] Sending connection context
0925 10:57:33.567378 (+ 140us) client_negotiation.cc:240] Negotiation successful
0925 10:57:33.567673 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260925 10:57:33.572893 13062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d: Bootstrap starting.
I20260925 10:57:33.576073 13062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d: Neither blocks nor log segments found. Creating new log.
I20260925 10:57:33.577160 13062 log.cc:824] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:33.579877 13062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d: No bootstrap required, opened a new log
I20260925 10:57:33.588061 13062 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } }
I20260925 10:57:33.588413 13062 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:57:33.588522 13062 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6db3cb9a53f64268adcca95e2c1fce3d, State: Initialized, Role: FOLLOWER
I20260925 10:57:33.589120 13062 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } }
I20260925 10:57:33.589970 13238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } } }
I20260925 10:57:33.590443 13238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 10:57:33.591295 13062 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d [sys.catalog]: configured and running, proceeding with master startup.
W20260925 10:57:33.602927 13249 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 10:57:33.603071 13249 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 10:57:33.602110 12995 master_runner.cc:186] Error getting master registration for 127.31.62.189:35409: OK, instance_id { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" instance_seqno: 1790333852890419 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 10:57:33.614989 12995 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 10:57:33.618404 13136 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } attempt: 5
W20260925 10:57:33.628723 13165 tablet.cc:2369] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 10:57:33 dist-test-slave-nx7s krb5kdc[12982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333852, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.189@KRBTEST.COM for kudu/127.31.62.188@KRBTEST.COM
I20260925 10:57:33.637331 13139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.619085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:40939 (local address 127.0.0.1:41960)
0925 10:57:33.619374 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:33.619405 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:33.619539 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:33.619934 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:33.619939 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:33.620496 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:33.620705 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:33.620719 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.622312 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.622318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:33.623706 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:33.623722 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.624881 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.624887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:33.625117 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.625768 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:33.625790 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:33.630254 (+ 4464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:33.634389 (+ 4135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:33.634395 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:33.634400 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:33.635009 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:33.635424 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:33.635430 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:33.635433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:33.635624 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:33.636249 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:33.636255 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:33.636421 (+ 166us) client_negotiation.cc:769] Sending connection context
0925 10:57:33.636600 (+ 179us) client_negotiation.cc:240] Negotiation successful
0925 10:57:33.636939 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260925 10:57:33.637370 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:33.619195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41960 (local address 127.31.62.188:40939)
0925 10:57:33.619403 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:33.619409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:33.619466 (+ 57us) server_negotiation.cc:407] Connection header received
0925 10:57:33.619675 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:33.619681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:33.619767 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:33.619876 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:33.620957 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.622144 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.623885 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:33.624616 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:33.624746 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:33.630439 (+ 5693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:33.630446 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:33.630451 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:33.630497 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:33.634207 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.635154 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.635160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.635164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.635313 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:33.635735 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:33.635740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:33.635744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:33.636128 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:33.636294 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:33.636712 (+ 418us) server_negotiation.cc:299] Negotiation successful
0925 10:57:33.636980 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260925 10:57:33.643599 13136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc: Bootstrap starting.
W20260925 10:57:33.645812 13016 tablet.cc:2369] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:57:33.649834 13136 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc: Neither blocks nor log segments found. Creating new log.
I20260925 10:57:33.651193 13136 log.cc:824] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:33.654639 13136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc: No bootstrap required, opened a new log
I20260925 10:57:33.665633 13136 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } }
I20260925 10:57:33.666059 13136 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:57:33.666188 13136 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68800bc00c714ff48f16845ebeeb08bc, State: Initialized, Role: FOLLOWER
I20260925 10:57:33.666885 13136 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } }
I20260925 10:57:33.667840 13252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } } }
I20260925 10:57:33.668277 13252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc [sys.catalog]: This master's current role is: FOLLOWER
I20260925 10:57:33.669265 13136 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:57:33.688288 13069 master_runner.cc:263] No leader master found from master 68800bc00c714ff48f16845ebeeb08bc
I20260925 10:57:33.688447 13069 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 10:57:33.690929 13263 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 10:57:33.691075 13263 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 10:57:33.697492 13224 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:57:33.697814 13224 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } }
I20260925 10:57:33.699455 13224 leader_election.cc:288] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6db3cb9a53f64268adcca95e2c1fce3d (127.31.62.190:44061), 68800bc00c714ff48f16845ebeeb08bc (127.31.62.189:35409)
I20260925 10:57:33.701052 13111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2bee0749cbc4dff94f798be44003929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68800bc00c714ff48f16845ebeeb08bc" is_pre_election: true
I20260925 10:57:33.701710 13111 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2bee0749cbc4dff94f798be44003929 in term 0.
I20260925 10:57:33.705566 13161 leader_election.cc:302] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68800bc00c714ff48f16845ebeeb08bc, b2bee0749cbc4dff94f798be44003929; no voters:
I20260925 10:57:33.706102 13224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:57:33.706257 13224 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:57:33.706347 13224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:57:33.707101 13037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2bee0749cbc4dff94f798be44003929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" is_pre_election: true
I20260925 10:57:33.709014 13224 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } }
I20260925 10:57:33.709349 13037 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2bee0749cbc4dff94f798be44003929 in term 0.
I20260925 10:57:33.710129 13037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2bee0749cbc4dff94f798be44003929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6db3cb9a53f64268adcca95e2c1fce3d"
I20260925 10:57:33.710242 13224 leader_election.cc:288] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [CANDIDATE]: Term 1 election: Requested vote from peers 6db3cb9a53f64268adcca95e2c1fce3d (127.31.62.190:44061), 68800bc00c714ff48f16845ebeeb08bc (127.31.62.189:35409)
I20260925 10:57:33.710318 13037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:57:33.710866 13111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2bee0749cbc4dff94f798be44003929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68800bc00c714ff48f16845ebeeb08bc"
I20260925 10:57:33.711073 13111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:57:33.713513 13037 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2bee0749cbc4dff94f798be44003929 in term 1.
I20260925 10:57:33.713975 13162 leader_election.cc:302] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6db3cb9a53f64268adcca95e2c1fce3d, b2bee0749cbc4dff94f798be44003929; no voters:
I20260925 10:57:33.714362 13224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:57:33.715205 13111 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2bee0749cbc4dff94f798be44003929 in term 1.
I20260925 10:57:33.725138 13224 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [term 1 LEADER]: Becoming Leader. State: Replica: b2bee0749cbc4dff94f798be44003929, State: Running, Role: LEADER
I20260925 10:57:33.725827 13224 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } }
I20260925 10:57:33.739068 13265 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2bee0749cbc4dff94f798be44003929. Latest consensus state: current_term: 1 leader_uuid: "b2bee0749cbc4dff94f798be44003929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } } }
I20260925 10:57:33.739511 13265 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [sys.catalog]: This master's current role is: LEADER
I20260925 10:57:33.752080 13267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:57:33.754459 13267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:57:33.774603 13111 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc [term 1 FOLLOWER]: Refusing update from remote peer b2bee0749cbc4dff94f798be44003929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:57:33.775815 13265 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:57:33.783545 13037 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d [term 1 FOLLOWER]: Refusing update from remote peer b2bee0749cbc4dff94f798be44003929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:57:33.788106 13265 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:57:33.818295 13238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2bee0749cbc4dff94f798be44003929. Latest consensus state: current_term: 1 leader_uuid: "b2bee0749cbc4dff94f798be44003929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } } }
I20260925 10:57:33.818642 13238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 10:57:33.817405 13252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2bee0749cbc4dff94f798be44003929. Latest consensus state: current_term: 1 leader_uuid: "b2bee0749cbc4dff94f798be44003929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } } }
I20260925 10:57:33.820134 13252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc [sys.catalog]: This master's current role is: FOLLOWER
I20260925 10:57:33.824987 13270 mvcc.cc:205] Tried to move back new op lower bound from 7333207465048838144 to 7333207464871694336. Current Snapshot: MvccSnapshot[applied={T|T < 7333207465048838144}]
I20260925 10:57:33.827301 13268 mvcc.cc:205] Tried to move back new op lower bound from 7333207465048838144 to 7333207464871694336. Current Snapshot: MvccSnapshot[applied={T|T < 7333207465048838144}]
I20260925 10:57:33.834359 13269 mvcc.cc:205] Tried to move back new op lower bound from 7333207465048838144 to 7333207464871694336. Current Snapshot: MvccSnapshot[applied={T|T < 7333207465048838144}]
I20260925 10:57:33.839882 13224 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b2bee0749cbc4dff94f798be44003929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } } }
I20260925 10:57:33.840291 13224 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [sys.catalog]: This master's current role is: LEADER
I20260925 10:57:33.840816 13224 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b2bee0749cbc4dff94f798be44003929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } } }
I20260925 10:57:33.841086 13224 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929 [sys.catalog]: This master's current role is: LEADER
I20260925 10:57:33.841367 13252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b2bee0749cbc4dff94f798be44003929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } } }
I20260925 10:57:33.841614 13252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc [sys.catalog]: This master's current role is: FOLLOWER
I20260925 10:57:33.845196 13267 catalog_manager.cc:1347] Generated new cluster ID: accc8319ed9e419497d16b6c0f00389b
I20260925 10:57:33.845302 13267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:57:33.859028 13238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b2bee0749cbc4dff94f798be44003929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db3cb9a53f64268adcca95e2c1fce3d" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44061 } } peers { permanent_uuid: "68800bc00c714ff48f16845ebeeb08bc" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 35409 } } peers { permanent_uuid: "b2bee0749cbc4dff94f798be44003929" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 40939 } } }
I20260925 10:57:33.859323 13238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 10:57:33.875663 13267 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:57:33.877686 13267 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:57:33.896162 13267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2bee0749cbc4dff94f798be44003929: Generated new TSK 0
I20260925 10:57:33.897876 13267 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:57:34.605131 13249 catalog_manager.cc:1259] Loaded cluster ID: accc8319ed9e419497d16b6c0f00389b
I20260925 10:57:34.605234 13249 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d: loading cluster ID for follower catalog manager: success
I20260925 10:57:34.607683 13249 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d: acquiring CA information for follower catalog manager: success
I20260925 10:57:34.609484 13249 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6db3cb9a53f64268adcca95e2c1fce3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 10:57:34.693115 13263 catalog_manager.cc:1259] Loaded cluster ID: accc8319ed9e419497d16b6c0f00389b
I20260925 10:57:34.693218 13263 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc: loading cluster ID for follower catalog manager: success
I20260925 10:57:34.695631 13263 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc: acquiring CA information for follower catalog manager: success
I20260925 10:57:34.697430 13263 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 68800bc00c714ff48f16845ebeeb08bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 10:57:35.037757 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 12995
I20260925 10:57:35.038996 13057 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:35.149233 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 12995
I20260925 10:57:35.165884 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 13069
I20260925 10:57:35.166915 13131 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:35.272684 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 13069
I20260925 10:57:35.289636 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 13143
I20260925 10:57:35.290697 13206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:35.400379 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 13143
2026-09-25T10:57:35Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4651 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:57:35 dist-test-slave-nx7s krb5kdc[13287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:57:35 dist-test-slave-nx7s krb5kdc[13287](info): set up 2 sockets
Sep 25 10:57:35 dist-test-slave-nx7s krb5kdc[13287](info): commencing operation
krb5kdc: starting...
W20260925 10:57:36.405673 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.968s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:57:36 dist-test-slave-nx7s krb5kdc[13287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:57:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:57:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:57:36.629093 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:36245
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:37377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:36245
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:57:36.956451 13300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:36.956787 13300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:36.956933 13300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:36.964697 13300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:57:36.964865 13300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:36.964941 13300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:36.965008 13300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:57:36.965054 13300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:57:36.975313 13300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:37377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:36245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:36245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:36.976611 13300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:36.977955 13300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:36.994813 13305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:36.994755 13308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:36.994791 13306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:36.995549 13300 server_base.cc:1034] running on GCE node
I20260925 10:57:36.996368 13300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:36.997906 13300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:36.999123 13300 hybrid_clock.cc:648] HybridClock initialized: now 1790333856999064 us; error 40 us; skew 500 ppm
I20260925 10:57:36.999471 13300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:36 dist-test-slave-nx7s krb5kdc[13287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333856, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:37.002632 13300 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:57:37.004092 13300 webserver.cc:460] Webserver started at http://127.31.62.190:40487/ using document root <none> and password file <none>
I20260925 10:57:37.004805 13300 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:37.004951 13300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:37.005250 13300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:57:37.008409 13300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "40168bf2d054461e9a34934aefec996c"
format_stamp: "Formatted at 2026-09-25 10:57:37 on dist-test-slave-nx7s"
server_key: "f6e688650b3f378ae14e01c87e24b020"
server_key_iv: "da5155e81c6831ab09f9c30497c9de98"
server_key_version: "encryptionkey@0"
I20260925 10:57:37.009230 13300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "40168bf2d054461e9a34934aefec996c"
format_stamp: "Formatted at 2026-09-25 10:57:37 on dist-test-slave-nx7s"
server_key: "f6e688650b3f378ae14e01c87e24b020"
server_key_iv: "da5155e81c6831ab09f9c30497c9de98"
server_key_version: "encryptionkey@0"
I20260925 10:57:37.014544 13300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:57:37.018154 13315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:37.019558 13300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:57:37.019721 13300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "40168bf2d054461e9a34934aefec996c"
format_stamp: "Formatted at 2026-09-25 10:57:37 on dist-test-slave-nx7s"
server_key: "f6e688650b3f378ae14e01c87e24b020"
server_key_iv: "da5155e81c6831ab09f9c30497c9de98"
server_key_version: "encryptionkey@0"
I20260925 10:57:37.019905 13300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:37.039234 13300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:37.043500 13300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:37.043779 13300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:37.060791 13300 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:36245
I20260925 10:57:37.060829 13366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:36245 every 8 connection(s)
I20260925 10:57:37.062242 13300 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:57:37.066010 13367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:57:37.070354 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 13300
I20260925 10:57:37.070554 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:57:37.071197 31994 external_mini_cluster.cc:1428] Setting key dccca24f21151da0cb642be2540e9a0a
I20260925 10:57:37.077448 13367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c: Bootstrap starting.
I20260925 10:57:37.081204 13367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c: Neither blocks nor log segments found. Creating new log.
I20260925 10:57:37.082399 13367 log.cc:824] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:37.085245 13367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c: No bootstrap required, opened a new log
Sep 25 10:57:37 dist-test-slave-nx7s krb5kdc[13287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:37.093497 13367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40168bf2d054461e9a34934aefec996c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36245 } }
I20260925 10:57:37.093925 13367 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:57:37.094033 13367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40168bf2d054461e9a34934aefec996c, State: Initialized, Role: FOLLOWER
I20260925 10:57:37.094338 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:37.073111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50354 (local address 127.31.62.190:36245)
0925 10:57:37.073876 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:37.073891 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:37.073933 (+ 42us) server_negotiation.cc:407] Connection header received
0925 10:57:37.075889 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:37.075914 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:37.076327 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:37.076665 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:37.078534 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:37.079914 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:37.081803 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:37.082540 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:37.082711 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:37.086695 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:37.086703 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:37.086722 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:37.086762 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:37.089912 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:37.090715 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:37.090723 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:37.090730 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:37.090921 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:37.091315 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:37.091319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:37.091324 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:37.091918 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:37.092219 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:37.092673 (+ 454us) server_negotiation.cc:299] Negotiation successful
0925 10:57:37.093151 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":350,"threads_started":1}
I20260925 10:57:37.094717 13367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40168bf2d054461e9a34934aefec996c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36245 } }
I20260925 10:57:37.094980 13367 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:57:37.095115 13367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:57:37.095276 13367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:57:37.098548 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:57:37.098484 13367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40168bf2d054461e9a34934aefec996c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36245 } }
I20260925 10:57:37.099026 13367 leader_election.cc:302] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40168bf2d054461e9a34934aefec996c; no voters:
I20260925 10:57:37.099661 13367 leader_election.cc:288] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:57:37.099761 13372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:57:37.100939 13372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [term 1 LEADER]: Becoming Leader. State: Replica: 40168bf2d054461e9a34934aefec996c, State: Running, Role: LEADER
I20260925 10:57:37.101619 13372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40168bf2d054461e9a34934aefec996c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36245 } }
I20260925 10:57:37.102108 13367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:57:37.104733 13374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40168bf2d054461e9a34934aefec996c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40168bf2d054461e9a34934aefec996c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36245 } } }
I20260925 10:57:37.105185 13374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [sys.catalog]: This master's current role is: LEADER
I20260925 10:57:37.105610 13373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40168bf2d054461e9a34934aefec996c. Latest consensus state: current_term: 1 leader_uuid: "40168bf2d054461e9a34934aefec996c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40168bf2d054461e9a34934aefec996c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36245 } } }
I20260925 10:57:37.105924 13373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c [sys.catalog]: This master's current role is: LEADER
I20260925 10:57:37.107261 13381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:57:37.112465 13381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:57:37.122016 13381 catalog_manager.cc:1347] Generated new cluster ID: f388b3f6faff4742b86ac684edcda7b3
I20260925 10:57:37.122148 13381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:57:37.139743 13381 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:57:37.141669 13381 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:57:37.154917 13381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40168bf2d054461e9a34934aefec996c: Generated new TSK 0
I20260925 10:57:37.155864 13381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:57:37.211258 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 13300
I20260925 10:57:37.212625 13362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:37.309641 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 13300
2026-09-25T10:57:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1905 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:57:37 dist-test-slave-nx7s krb5kdc[13395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:57:37 dist-test-slave-nx7s krb5kdc[13395](info): set up 2 sockets
Sep 25 10:57:37 dist-test-slave-nx7s krb5kdc[13395](info): commencing operation
krb5kdc: starting...
W20260925 10:57:38.321650 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:57:38 dist-test-slave-nx7s krb5kdc[13395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:57:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:57:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:57:38.548820 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41871
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:41871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:57:38.873333 13408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:38.873631 13408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:38.873761 13408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:38.881595 13408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:57:38.881708 13408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:38.881779 13408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:38.881857 13408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:57:38.881932 13408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:57:38.892586 13408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:41871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:38.894042 13408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:38.895411 13408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:38.907012 13413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:38.906970 13414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:38.906920 13416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:38.907797 13408 server_base.cc:1034] running on GCE node
I20260925 10:57:38.908593 13408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:38.910202 13408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:38.911434 13408 hybrid_clock.cc:648] HybridClock initialized: now 1790333858911381 us; error 42 us; skew 500 ppm
I20260925 10:57:38.911805 13408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:38 dist-test-slave-nx7s krb5kdc[13395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333858, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:38.920058 13408 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:57:38.921818 13408 webserver.cc:460] Webserver started at http://127.31.62.190:44107/ using document root <none> and password file <none>
I20260925 10:57:38.922537 13408 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:38.922646 13408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:38.922910 13408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:57:38.926180 13408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "0e68e89e7e734035b161024a74256219"
format_stamp: "Formatted at 2026-09-25 10:57:38 on dist-test-slave-nx7s"
server_key: "9fd4bda4585a30c38d63d86c082ae080"
server_key_iv: "ced6b6c69ba096503111472cb2974fb6"
server_key_version: "encryptionkey@0"
I20260925 10:57:38.926972 13408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "0e68e89e7e734035b161024a74256219"
format_stamp: "Formatted at 2026-09-25 10:57:38 on dist-test-slave-nx7s"
server_key: "9fd4bda4585a30c38d63d86c082ae080"
server_key_iv: "ced6b6c69ba096503111472cb2974fb6"
server_key_version: "encryptionkey@0"
I20260925 10:57:38.932468 13408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 10:57:38.936267 13423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:38.937809 13408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:57:38.937999 13408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "0e68e89e7e734035b161024a74256219"
format_stamp: "Formatted at 2026-09-25 10:57:38 on dist-test-slave-nx7s"
server_key: "9fd4bda4585a30c38d63d86c082ae080"
server_key_iv: "ced6b6c69ba096503111472cb2974fb6"
server_key_version: "encryptionkey@0"
I20260925 10:57:38.938206 13408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:38.956372 13408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:38.960706 13408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:38.961027 13408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:38.978646 13408 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:41871
I20260925 10:57:38.978695 13474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:41871 every 8 connection(s)
I20260925 10:57:38.980012 13408 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:57:38.980432 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 13408
I20260925 10:57:38.980626 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:57:38.981478 31994 external_mini_cluster.cc:1428] Setting key b5fe978e72701ae9a749f2462200caaa
I20260925 10:57:38.984442 13475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 10:57:38 dist-test-slave-nx7s krb5kdc[13395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:38.995683 13475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: Bootstrap starting.
I20260925 10:57:38.999336 13475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: Neither blocks nor log segments found. Creating new log.
I20260925 10:57:39.000470 13475 log.cc:824] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:39.003866 13475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: No bootstrap required, opened a new log
I20260925 10:57:39.004431 13478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:38.983445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44990 (local address 127.31.62.190:41871)
0925 10:57:38.984476 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:38.984490 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:38.984558 (+ 68us) server_negotiation.cc:407] Connection header received
0925 10:57:38.986221 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:38.986243 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:38.986644 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:38.987017 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:38.988493 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:38.989939 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:38.991254 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:38.991955 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:38.992143 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:38.996533 (+ 4390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:38.996542 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:38.996560 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:38.996601 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:38.999936 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:39.000709 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:39.000717 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:39.000725 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:39.000969 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:39.001476 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:39.001481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:39.001484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:39.002150 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:39.002378 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:39.002885 (+ 507us) server_negotiation.cc:299] Negotiation successful
0925 10:57:39.003331 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":612,"threads_started":1}
I20260925 10:57:39.012123 13475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } }
I20260925 10:57:39.012507 13475 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:57:39.012614 13475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e68e89e7e734035b161024a74256219, State: Initialized, Role: FOLLOWER
I20260925 10:57:39.013242 13475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } }
I20260925 10:57:39.013473 13475 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:57:39.013605 13475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:57:39.013757 13475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:57:39.015889 13475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } }
I20260925 10:57:39.016396 13475 leader_election.cc:302] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e68e89e7e734035b161024a74256219; no voters:
I20260925 10:57:39.017191 13475 leader_election.cc:288] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:57:39.017288 13480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:57:39.017957 13480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 1 LEADER]: Becoming Leader. State: Replica: 0e68e89e7e734035b161024a74256219, State: Running, Role: LEADER
I20260925 10:57:39.018538 13480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } }
I20260925 10:57:39.019240 13475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:57:39.021258 13482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e68e89e7e734035b161024a74256219. Latest consensus state: current_term: 1 leader_uuid: "0e68e89e7e734035b161024a74256219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } } }
I20260925 10:57:39.021633 13482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [sys.catalog]: This master's current role is: LEADER
I20260925 10:57:39.021100 13481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e68e89e7e734035b161024a74256219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } } }
I20260925 10:57:39.022336 13481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [sys.catalog]: This master's current role is: LEADER
I20260925 10:57:39.028133 13488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:57:39.033366 13488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:57:39.043083 13488 catalog_manager.cc:1347] Generated new cluster ID: 2be48f33445145cea72e0353d86b563c
I20260925 10:57:39.043190 13488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:57:39.074596 13488 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:57:39.076550 13488 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:57:39.093545 13488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: Generated new TSK 0
I20260925 10:57:39.094436 13488 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:57:39.144413 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41871
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:57:39.477631 13501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:39.477957 13501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:39.478066 13501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:39.485548 13501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:39.485663 13501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:39.485782 13501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:57:39.495515 13501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:39.496778 13501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:39.498092 13501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:39.510823 13506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:39.510907 13507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:39.510848 13509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:39.511857 13501 server_base.cc:1034] running on GCE node
I20260925 10:57:39.512440 13501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:39.513401 13501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:39.514612 13501 hybrid_clock.cc:648] HybridClock initialized: now 1790333859514558 us; error 43 us; skew 500 ppm
I20260925 10:57:39.514928 13501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:39 dist-test-slave-nx7s krb5kdc[13395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333859, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:39.518059 13501 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:57:39.519557 13501 webserver.cc:460] Webserver started at http://127.31.62.129:44759/ using document root <none> and password file <none>
I20260925 10:57:39.520244 13501 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:39.520365 13501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:39.520650 13501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:57:39.523597 13501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "5423854314a740c5a7f9d497df7e9778"
format_stamp: "Formatted at 2026-09-25 10:57:39 on dist-test-slave-nx7s"
server_key: "a2d1be3111f43a29333c2698ac8d6ee5"
server_key_iv: "7056ee984f8c4dcf4843ef67fee40a7e"
server_key_version: "encryptionkey@0"
I20260925 10:57:39.524390 13501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "5423854314a740c5a7f9d497df7e9778"
format_stamp: "Formatted at 2026-09-25 10:57:39 on dist-test-slave-nx7s"
server_key: "a2d1be3111f43a29333c2698ac8d6ee5"
server_key_iv: "7056ee984f8c4dcf4843ef67fee40a7e"
server_key_version: "encryptionkey@0"
I20260925 10:57:39.529862 13501 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 10:57:39.533339 13516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:39.534794 13501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260925 10:57:39.534948 13501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "5423854314a740c5a7f9d497df7e9778"
format_stamp: "Formatted at 2026-09-25 10:57:39 on dist-test-slave-nx7s"
server_key: "a2d1be3111f43a29333c2698ac8d6ee5"
server_key_iv: "7056ee984f8c4dcf4843ef67fee40a7e"
server_key_version: "encryptionkey@0"
I20260925 10:57:39.535132 13501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:39.550163 13501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:39.553994 13501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:39.554261 13501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:39.555197 13501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:57:39.556521 13501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:57:39.556600 13501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:39.556735 13501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:57:39.556794 13501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:39.586839 13501 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:34229
I20260925 10:57:39.586875 13628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:34229 every 8 connection(s)
I20260925 10:57:39.588104 13501 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:57:39.595904 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 13501
I20260925 10:57:39.596117 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:57:39.597018 31994 external_mini_cluster.cc:1428] Setting key 88fb941b3bde100319160cb286a744cf
Sep 25 10:57:39 dist-test-slave-nx7s krb5kdc[13395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333859, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:39.609102 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:39.591023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:41943 (local address 127.31.62.190:41871)
0925 10:57:39.591428 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:39.591433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:39.592516 (+ 1083us) server_negotiation.cc:407] Connection header received
0925 10:57:39.594360 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:39.594366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:39.594461 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:39.594558 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:39.596214 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:39.597284 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:39.599059 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:39.599634 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:39.599763 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:39.603466 (+ 3703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:39.603470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:39.603474 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:39.603506 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:39.605923 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:39.606773 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:39.606776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:39.606779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:39.606895 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:39.607269 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:39.607272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:39.607273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:39.607519 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:39.607650 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:39.608478 (+ 828us) server_negotiation.cc:299] Negotiation successful
0925 10:57:39.608704 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":191,"threads_started":1}
I20260925 10:57:39.610074 13632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:39.591623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:41943)
0925 10:57:39.592322 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:39.592386 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:39.594083 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:39.594838 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:39.594846 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:39.595417 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:39.596004 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:39.596022 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:39.597445 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:39.597449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:39.598923 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:39.598935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:39.599821 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:39.599826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:39.600022 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:39.600664 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:39.600686 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:39.603243 (+ 2557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:39.606065 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:39.606075 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:39.606094 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:39.606672 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:39.606990 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:39.606994 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:39.606998 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:39.607190 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:39.607658 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:39.607666 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:39.608056 (+ 390us) client_negotiation.cc:769] Sending connection context
0925 10:57:39.608361 (+ 305us) client_negotiation.cc:240] Negotiation successful
0925 10:57:39.608938 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":269,"threads_started":1}
I20260925 10:57:39.611605 13629 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41871
I20260925 10:57:39.611968 13629 heartbeater.cc:464] Registering TS with master...
I20260925 10:57:39.612950 13629 heartbeater.cc:511] Master 127.31.62.190:41871 requested a full tablet report, sending...
I20260925 10:57:39.615156 13440 ts_manager.cc:194] Registered new tserver with Master: 5423854314a740c5a7f9d497df7e9778 (127.31.62.129:34229)
I20260925 10:57:39.616605 13440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:41943
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:57:39.648407 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41871
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:57:39.970700 13635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:39.971002 13635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:39.971110 13635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:39.978607 13635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:39.978724 13635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:39.978881 13635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:57:39.988759 13635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:39.990104 13635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:39.991446 13635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:40.004573 13640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:40.004500 13641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:40.005236 13643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:40.005228 13635 server_base.cc:1034] running on GCE node
I20260925 10:57:40.005936 13635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:40.007017 13635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:40.008235 13635 hybrid_clock.cc:648] HybridClock initialized: now 1790333860008185 us; error 44 us; skew 500 ppm
I20260925 10:57:40.008565 13635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:40 dist-test-slave-nx7s krb5kdc[13395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333860, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:40.011763 13635 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:57:40.013325 13635 webserver.cc:460] Webserver started at http://127.31.62.130:43865/ using document root <none> and password file <none>
I20260925 10:57:40.014063 13635 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:40.014201 13635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:40.014523 13635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:57:40.017629 13635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "7a4510b7224f42049881f3d65bd68685"
format_stamp: "Formatted at 2026-09-25 10:57:40 on dist-test-slave-nx7s"
server_key: "71214e07852e7309d7bd50b9182185fd"
server_key_iv: "ca54fa1bceaa23f1a7b9d8992ea9f039"
server_key_version: "encryptionkey@0"
I20260925 10:57:40.018415 13635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "7a4510b7224f42049881f3d65bd68685"
format_stamp: "Formatted at 2026-09-25 10:57:40 on dist-test-slave-nx7s"
server_key: "71214e07852e7309d7bd50b9182185fd"
server_key_iv: "ca54fa1bceaa23f1a7b9d8992ea9f039"
server_key_version: "encryptionkey@0"
I20260925 10:57:40.023990 13635 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 10:57:40.027828 13650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:40.029433 13635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:57:40.029603 13635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "7a4510b7224f42049881f3d65bd68685"
format_stamp: "Formatted at 2026-09-25 10:57:40 on dist-test-slave-nx7s"
server_key: "71214e07852e7309d7bd50b9182185fd"
server_key_iv: "ca54fa1bceaa23f1a7b9d8992ea9f039"
server_key_version: "encryptionkey@0"
I20260925 10:57:40.029803 13635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:40.044118 13635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:40.048255 13635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:40.048525 13635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:40.049506 13635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:57:40.050824 13635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:57:40.050894 13635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:40.051002 13635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:57:40.051054 13635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:40.081072 13635 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:32815
I20260925 10:57:40.081105 13762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:32815 every 8 connection(s)
I20260925 10:57:40.082420 13635 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:57:40.089979 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 13635
I20260925 10:57:40.090152 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:57:40.090579 31994 external_mini_cluster.cc:1428] Setting key 5b0b642daf045923fd977a93320bafd7
Sep 25 10:57:40 dist-test-slave-nx7s krb5kdc[13395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333860, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:40.103142 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.084990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:58029 (local address 127.31.62.190:41871)
0925 10:57:40.085201 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:40.085208 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:40.086416 (+ 1208us) server_negotiation.cc:407] Connection header received
0925 10:57:40.088135 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:40.088139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:40.088217 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:40.088307 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:40.089876 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.090703 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.092648 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.093091 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.093184 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.097156 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:40.097161 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:40.097164 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:40.097198 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:40.099795 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:40.100573 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:40.100580 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:40.100583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:40.100771 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:40.101247 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:40.101253 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:40.101256 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:40.101663 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:40.101819 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:40.102486 (+ 667us) server_negotiation.cc:299] Negotiation successful
0925 10:57:40.102750 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 10:57:40.103886 13765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.085563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:58029)
0925 10:57:40.086260 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:40.086298 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:40.087896 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:40.088505 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:40.088536 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:40.089072 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:40.089700 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:40.089725 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.090858 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.090863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:40.092485 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:40.092497 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.093301 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.093308 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:40.093589 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.094276 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:40.094305 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:40.096918 (+ 2613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:40.099963 (+ 3045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:40.099971 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:40.099986 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:40.100433 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:40.100935 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:40.100938 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:40.100941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:40.101131 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:40.101786 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:40.101794 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:40.102125 (+ 331us) client_negotiation.cc:769] Sending connection context
0925 10:57:40.102343 (+ 218us) client_negotiation.cc:240] Negotiation successful
0925 10:57:40.102759 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":266,"threads_started":1}
I20260925 10:57:40.105396 13763 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41871
I20260925 10:57:40.105772 13763 heartbeater.cc:464] Registering TS with master...
I20260925 10:57:40.106829 13763 heartbeater.cc:511] Master 127.31.62.190:41871 requested a full tablet report, sending...
I20260925 10:57:40.108475 13440 ts_manager.cc:194] Registered new tserver with Master: 7a4510b7224f42049881f3d65bd68685 (127.31.62.130:32815)
I20260925 10:57:40.109319 13440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:58029
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:57:40.144379 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41871
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:57:40.477439 13768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:40.477738 13768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:40.477856 13768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:40.485687 13768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:40.485808 13768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:40.485965 13768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:57:40.495748 13768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:41871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:40.497087 13768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:40.498399 13768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:40.511296 13773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:40.511346 13774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:40.511320 13776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:40.512499 13768 server_base.cc:1034] running on GCE node
I20260925 10:57:40.513235 13768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:40.514348 13768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:40.515566 13768 hybrid_clock.cc:648] HybridClock initialized: now 1790333860515510 us; error 47 us; skew 500 ppm
I20260925 10:57:40.515903 13768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:40 dist-test-slave-nx7s krb5kdc[13395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333860, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:40.519203 13768 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:57:40.520778 13768 webserver.cc:460] Webserver started at http://127.31.62.131:41849/ using document root <none> and password file <none>
I20260925 10:57:40.521512 13768 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:40.521621 13768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:40.521919 13768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:57:40.524995 13768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "38921275dbb34d378e099ae3b33fe899"
format_stamp: "Formatted at 2026-09-25 10:57:40 on dist-test-slave-nx7s"
server_key: "c05ceadf40192d675f121e371e0ab0b1"
server_key_iv: "77c5ed51922dba831d0fe306a154b304"
server_key_version: "encryptionkey@0"
I20260925 10:57:40.525875 13768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "38921275dbb34d378e099ae3b33fe899"
format_stamp: "Formatted at 2026-09-25 10:57:40 on dist-test-slave-nx7s"
server_key: "c05ceadf40192d675f121e371e0ab0b1"
server_key_iv: "77c5ed51922dba831d0fe306a154b304"
server_key_version: "encryptionkey@0"
I20260925 10:57:40.531490 13768 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:57:40.535312 13783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:40.536793 13768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:57:40.536963 13768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "38921275dbb34d378e099ae3b33fe899"
format_stamp: "Formatted at 2026-09-25 10:57:40 on dist-test-slave-nx7s"
server_key: "c05ceadf40192d675f121e371e0ab0b1"
server_key_iv: "77c5ed51922dba831d0fe306a154b304"
server_key_version: "encryptionkey@0"
I20260925 10:57:40.537164 13768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:40.555678 13768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:40.559984 13768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:40.560272 13768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:40.561342 13768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:57:40.562839 13768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:57:40.562916 13768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:40.563040 13768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:57:40.563091 13768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:40.593411 13768 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:42007
I20260925 10:57:40.593478 13895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:42007 every 8 connection(s)
I20260925 10:57:40.594693 13768 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:57:40.595923 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 13768
I20260925 10:57:40.596134 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:57:40.596746 31994 external_mini_cluster.cc:1428] Setting key ea76c0f56a33074d7538341d34209a9b
Sep 25 10:57:40 dist-test-slave-nx7s krb5kdc[13395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333860, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:40.615139 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.597624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:45649 (local address 127.31.62.190:41871)
0925 10:57:40.597829 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:40.597836 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:40.599202 (+ 1366us) server_negotiation.cc:407] Connection header received
0925 10:57:40.601253 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:40.601259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:40.601348 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:40.601431 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:40.602989 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.603752 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.605432 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.605876 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.605999 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.609498 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:40.609503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:40.609507 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:40.609542 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:40.612046 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:40.612768 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:40.612771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:40.612774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:40.612944 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:40.613364 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:40.613368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:40.613371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:40.613683 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:40.613816 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:40.614543 (+ 727us) server_negotiation.cc:299] Negotiation successful
0925 10:57:40.614772 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 10:57:40.616175 13898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.598251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:45649)
0925 10:57:40.599063 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:40.599107 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:40.600995 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:40.601699 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:40.601708 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:40.602230 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:40.602821 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:40.602839 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.603914 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.603918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:40.605290 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:40.605303 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.606117 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.606125 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:40.606359 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.607010 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:40.607039 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:40.609279 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:40.612187 (+ 2908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:40.612197 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:40.612211 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:40.612645 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:40.613051 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:40.613056 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:40.613060 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:40.613248 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:40.613806 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:40.613815 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:40.614174 (+ 359us) client_negotiation.cc:769] Sending connection context
0925 10:57:40.614431 (+ 257us) client_negotiation.cc:240] Negotiation successful
0925 10:57:40.614982 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":303,"threads_started":1}
I20260925 10:57:40.617683 13896 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41871
I20260925 10:57:40.618000 13896 heartbeater.cc:464] Registering TS with master...
I20260925 10:57:40.618764 13896 heartbeater.cc:511] Master 127.31.62.190:41871 requested a full tablet report, sending...
I20260925 10:57:40.620358 13440 ts_manager.cc:194] Registered new tserver with Master: 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007)
I20260925 10:57:40.620532 13629 heartbeater.cc:503] Master 127.31.62.190:41871 was elected leader, sending a full tablet report...
I20260925 10:57:40.621388 13440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:45649
I20260925 10:57:40.624728 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:57:40.641940 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.629117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.31.62.190:41871)
0925 10:57:40.629313 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:40.629320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:40.629734 (+ 414us) server_negotiation.cc:407] Connection header received
0925 10:57:40.629906 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:40.629912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:40.629996 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:40.630081 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:40.631490 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.632227 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.633611 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.634030 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.634150 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.636624 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:40.636630 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:40.636633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:40.636669 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:40.639272 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:40.639926 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:40.639932 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:40.639935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:40.640071 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:40.640425 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:40.640430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:40.640432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:40.640762 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:40.641007 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:40.641366 (+ 359us) server_negotiation.cc:299] Negotiation successful
0925 10:57:40.641607 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 10:57:40.646200 13439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:57:40.653117 13439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:57:40.685698 13911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.666631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45694 (local address 127.31.62.130:32815)
0925 10:57:40.675036 (+ 8405us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:40.675045 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:40.677666 (+ 2621us) server_negotiation.cc:407] Connection header received
0925 10:57:40.677785 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:40.677794 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:40.678119 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:40.678352 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:57:40.679263 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.680634 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.683028 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.684489 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.684629 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.684843 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:40.685166 (+ 323us) server_negotiation.cc:299] Negotiation successful
0925 10:57:40.685351 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8243,"thread_start_us":8102,"threads_started":1}
I20260925 10:57:40.685768 13912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.666090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:32815 (local address 127.0.0.1:45694)
0925 10:57:40.677340 (+ 11250us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:40.677361 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:40.677508 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:40.678653 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:40.678661 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:40.678691 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:57:40.679078 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:40.679092 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.680772 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.680778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:40.682804 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:40.682820 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.684685 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.684691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:40.684930 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.684978 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 10:57:40.685098 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 10:57:40.685350 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10411,"mutex_wait_us":1,"thread_start_us":12053,"threads_started":1}
I20260925 10:57:40.686749 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.665342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34229 (local address 127.0.0.1:41834)
0925 10:57:40.666880 (+ 1538us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:40.666925 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:40.667102 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:40.679494 (+ 12392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:40.679500 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:40.679533 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:57:40.680001 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:40.680024 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.682038 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.682043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:40.683967 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:40.683981 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.685894 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.685901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:40.686081 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.686113 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 10:57:40.686184 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 10:57:40.686344 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1299,"mutex_wait_us":55,"thread_start_us":1135,"threads_started":1}
I20260925 10:57:40.687369 13913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.667706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40374 (local address 127.31.62.131:42007)
0925 10:57:40.677076 (+ 9370us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:40.677085 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:40.678720 (+ 1635us) server_negotiation.cc:407] Connection header received
0925 10:57:40.678828 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:40.678836 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:40.679125 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:40.679320 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:57:40.680008 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.681685 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.684320 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.685718 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.685830 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.686057 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:40.686851 (+ 794us) server_negotiation.cc:299] Negotiation successful
0925 10:57:40.687024 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9169,"thread_start_us":9399,"threads_started":1}
I20260925 10:57:40.687378 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.665911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41834 (local address 127.31.62.129:34229)
0925 10:57:40.678875 (+ 12964us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:40.678884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:40.678914 (+ 30us) server_negotiation.cc:407] Connection header received
0925 10:57:40.679025 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:40.679033 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:40.679241 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:40.680058 (+ 817us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:57:40.680347 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.681856 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.684132 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.685639 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.686554 (+ 915us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.686797 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:40.686871 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 10:57:40.687049 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12804,"thread_start_us":12676,"threads_started":1}
I20260925 10:57:40.687666 13910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.667352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:42007 (local address 127.0.0.1:40374)
0925 10:57:40.678446 (+ 11094us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:40.678465 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:40.678586 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:40.679320 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:40.679324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:40.679346 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:57:40.679803 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:40.679814 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.681829 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.681835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:40.684152 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:40.684167 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.686198 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.686204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:40.686387 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.686418 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 10:57:40.686505 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 10:57:40.686643 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10931,"mutex_wait_us":166,"thread_start_us":9003,"threads_started":1}
I20260925 10:57:40.690984 13831 tablet_service.cc:1467] Processing CreateTablet for tablet 53ea2d47923e47c69285637659f8aa16 (DEFAULT_TABLE table=test-table [id=3270f1c98e44466aa8ccdebf422458b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:57:40.691082 13564 tablet_service.cc:1467] Processing CreateTablet for tablet 53ea2d47923e47c69285637659f8aa16 (DEFAULT_TABLE table=test-table [id=3270f1c98e44466aa8ccdebf422458b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:57:40.691012 13698 tablet_service.cc:1467] Processing CreateTablet for tablet 53ea2d47923e47c69285637659f8aa16 (DEFAULT_TABLE table=test-table [id=3270f1c98e44466aa8ccdebf422458b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:57:40.692319 13831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53ea2d47923e47c69285637659f8aa16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:57:40.692386 13698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53ea2d47923e47c69285637659f8aa16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:57:40.692385 13564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53ea2d47923e47c69285637659f8aa16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:57:40.704311 13914 tablet_bootstrap.cc:492] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899: Bootstrap starting.
I20260925 10:57:40.709002 13914 tablet_bootstrap.cc:654] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899: Neither blocks nor log segments found. Creating new log.
I20260925 10:57:40.710018 13914 log.cc:824] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:40.710558 13916 tablet_bootstrap.cc:492] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685: Bootstrap starting.
I20260925 10:57:40.710927 13915 tablet_bootstrap.cc:492] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778: Bootstrap starting.
I20260925 10:57:40.714388 13915 tablet_bootstrap.cc:654] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778: Neither blocks nor log segments found. Creating new log.
I20260925 10:57:40.714640 13914 tablet_bootstrap.cc:492] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899: No bootstrap required, opened a new log
I20260925 10:57:40.714735 13916 tablet_bootstrap.cc:654] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685: Neither blocks nor log segments found. Creating new log.
I20260925 10:57:40.714973 13914 ts_tablet_manager.cc:1397] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.006s
I20260925 10:57:40.715638 13915 log.cc:824] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:40.715711 13916 log.cc:824] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:40.719537 13916 tablet_bootstrap.cc:492] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685: No bootstrap required, opened a new log
I20260925 10:57:40.719555 13915 tablet_bootstrap.cc:492] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778: No bootstrap required, opened a new log
I20260925 10:57:40.719825 13916 ts_tablet_manager.cc:1397] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260925 10:57:40.719825 13915 ts_tablet_manager.cc:1397] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260925 10:57:40.723693 13914 raft_consensus.cc:348] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } }
I20260925 10:57:40.724102 13914 raft_consensus.cc:374] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:57:40.724231 13914 raft_consensus.cc:729] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38921275dbb34d378e099ae3b33fe899, State: Initialized, Role: FOLLOWER
I20260925 10:57:40.724829 13914 consensus_queue.cc:260] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } }
I20260925 10:57:40.726002 13896 heartbeater.cc:503] Master 127.31.62.190:41871 was elected leader, sending a full tablet report...
I20260925 10:57:40.728053 13914 ts_tablet_manager.cc:1428] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899: Time spent starting tablet: real 0.013s user 0.004s sys 0.009s
I20260925 10:57:40.730430 13916 raft_consensus.cc:348] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } }
I20260925 10:57:40.730430 13915 raft_consensus.cc:348] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } }
I20260925 10:57:40.730885 13915 raft_consensus.cc:374] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:57:40.730885 13916 raft_consensus.cc:374] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:57:40.730985 13915 raft_consensus.cc:729] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5423854314a740c5a7f9d497df7e9778, State: Initialized, Role: FOLLOWER
I20260925 10:57:40.730985 13916 raft_consensus.cc:729] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a4510b7224f42049881f3d65bd68685, State: Initialized, Role: FOLLOWER
I20260925 10:57:40.731617 13916 consensus_queue.cc:260] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } }
I20260925 10:57:40.731622 13915 consensus_queue.cc:260] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } }
I20260925 10:57:40.733208 13763 heartbeater.cc:503] Master 127.31.62.190:41871 was elected leader, sending a full tablet report...
I20260925 10:57:40.733600 13915 ts_tablet_manager.cc:1428] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260925 10:57:40.733716 13916 ts_tablet_manager.cc:1428] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260925 10:57:40.800372 13922 raft_consensus.cc:482] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:57:40.800729 13922 raft_consensus.cc:504] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } }
I20260925 10:57:40.802376 13922 leader_election.cc:288] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5423854314a740c5a7f9d497df7e9778 (127.31.62.129:34229), 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007)
I20260925 10:57:40.809315 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.802799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:43937 (local address 127.31.62.129:34229)
0925 10:57:40.803013 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:40.803019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:40.803538 (+ 519us) server_negotiation.cc:407] Connection header received
0925 10:57:40.803746 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:40.803752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:40.803833 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:40.803921 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:57:40.804548 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.805491 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.807437 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.808346 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.808468 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.808522 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:40.808909 (+ 387us) server_negotiation.cc:299] Negotiation successful
0925 10:57:40.809079 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 10:57:40.809392 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.802790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34229 (local address 127.31.62.130:43937)
0925 10:57:40.803424 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:40.803440 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:40.803611 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:40.803985 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:40.803991 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:40.804023 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:57:40.804390 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:40.804403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.805623 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.805629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:40.807291 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:40.807303 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.808564 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.808571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:40.808727 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.808763 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 10:57:40.808891 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 10:57:40.809019 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":25,"thread_start_us":347,"threads_started":1}
I20260925 10:57:40.809645 13913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.803696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:53443 (local address 127.31.62.131:42007)
0925 10:57:40.803893 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:40.803898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:40.804234 (+ 336us) server_negotiation.cc:407] Connection header received
0925 10:57:40.804344 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:40.804349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:40.804425 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:40.804506 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:57:40.805100 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.805965 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.807812 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.808660 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.808807 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.808911 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:40.809261 (+ 350us) server_negotiation.cc:299] Negotiation successful
0925 10:57:40.809389 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 10:57:40.809801 13924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.803534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:42007 (local address 127.31.62.130:53443)
0925 10:57:40.804126 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:40.804139 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:40.804239 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:40.804549 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:40.804553 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:40.804577 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:57:40.804965 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:40.804977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.806139 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.806143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:40.807665 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:40.807679 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.808883 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.808889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:40.809093 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.809128 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 10:57:40.809216 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 10:57:40.809340 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":45,"thread_start_us":288,"threads_started":1}
I20260925 10:57:40.810509 13584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53ea2d47923e47c69285637659f8aa16" candidate_uuid: "7a4510b7224f42049881f3d65bd68685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5423854314a740c5a7f9d497df7e9778" is_pre_election: true
I20260925 10:57:40.810917 13851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53ea2d47923e47c69285637659f8aa16" candidate_uuid: "7a4510b7224f42049881f3d65bd68685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38921275dbb34d378e099ae3b33fe899" is_pre_election: true
I20260925 10:57:40.811161 13584 raft_consensus.cc:2454] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a4510b7224f42049881f3d65bd68685 in term 0.
I20260925 10:57:40.811434 13851 raft_consensus.cc:2454] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a4510b7224f42049881f3d65bd68685 in term 0.
I20260925 10:57:40.811868 13653 leader_election.cc:302] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5423854314a740c5a7f9d497df7e9778, 7a4510b7224f42049881f3d65bd68685; no voters:
I20260925 10:57:40.812222 13922 raft_consensus.cc:2781] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:57:40.812345 13922 raft_consensus.cc:482] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:57:40.812441 13922 raft_consensus.cc:3037] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:57:40.814765 13922 raft_consensus.cc:504] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } }
I20260925 10:57:40.815347 13922 leader_election.cc:288] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [CANDIDATE]: Term 1 election: Requested vote from peers 5423854314a740c5a7f9d497df7e9778 (127.31.62.129:34229), 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007)
I20260925 10:57:40.815850 13584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53ea2d47923e47c69285637659f8aa16" candidate_uuid: "7a4510b7224f42049881f3d65bd68685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5423854314a740c5a7f9d497df7e9778"
I20260925 10:57:40.815902 13851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53ea2d47923e47c69285637659f8aa16" candidate_uuid: "7a4510b7224f42049881f3d65bd68685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38921275dbb34d378e099ae3b33fe899"
I20260925 10:57:40.816095 13851 raft_consensus.cc:3037] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:57:40.816082 13584 raft_consensus.cc:3037] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:57:40.819352 13584 raft_consensus.cc:2454] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a4510b7224f42049881f3d65bd68685 in term 1.
I20260925 10:57:40.819341 13851 raft_consensus.cc:2454] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a4510b7224f42049881f3d65bd68685 in term 1.
I20260925 10:57:40.819869 13653 leader_election.cc:302] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5423854314a740c5a7f9d497df7e9778, 7a4510b7224f42049881f3d65bd68685; no voters:
I20260925 10:57:40.820233 13922 raft_consensus.cc:2781] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:57:40.820730 13922 raft_consensus.cc:686] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [term 1 LEADER]: Becoming Leader. State: Replica: 7a4510b7224f42049881f3d65bd68685, State: Running, Role: LEADER
I20260925 10:57:40.821290 13922 consensus_queue.cc:237] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } }
I20260925 10:57:40.826073 13439 catalog_manager.cc:5495] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a4510b7224f42049881f3d65bd68685 (127.31.62.130). New cstate: current_term: 1 leader_uuid: "7a4510b7224f42049881f3d65bd68685" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } health_report { overall_health: HEALTHY } } }
W20260925 10:57:40.834352 13764 tablet.cc:2369] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:57:40.840404 13630 tablet.cc:2369] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:57:40.846467 13897 tablet.cc:2369] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:57:40.869989 13911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.865114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45702 (local address 127.31.62.130:32815)
0925 10:57:40.865284 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:40.865288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:40.865306 (+ 18us) server_negotiation.cc:407] Connection header received
0925 10:57:40.865397 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:40.865401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:40.865479 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:40.865619 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:57:40.865925 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.866644 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.867968 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.868373 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.868464 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.868959 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:57:40.869499 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:57:40.869625 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:40.869702 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 10:57:40.869774 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 10:57:40.878306 13584 raft_consensus.cc:1261] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Refusing update from remote peer 7a4510b7224f42049881f3d65bd68685: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:57:40.878306 13851 raft_consensus.cc:1261] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899 [term 1 FOLLOWER]: Refusing update from remote peer 7a4510b7224f42049881f3d65bd68685: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:57:40.879383 13925 consensus_queue.cc:1035] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:57:40.879746 13922 consensus_queue.cc:1035] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:57:40.906847 13439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:57:40.907126 13439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:57:40.910908 13439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: Sending DeleteTablet for 3 replicas of tablet 53ea2d47923e47c69285637659f8aa16
I20260925 10:57:40.912173 13698 tablet_service.cc:1514] Processing DeleteTablet for tablet 53ea2d47923e47c69285637659f8aa16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:57:40 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:45694
I20260925 10:57:40.912173 13564 tablet_service.cc:1514] Processing DeleteTablet for tablet 53ea2d47923e47c69285637659f8aa16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:57:40 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:41834
I20260925 10:57:40.912217 13831 tablet_service.cc:1514] Processing DeleteTablet for tablet 53ea2d47923e47c69285637659f8aa16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:57:40 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:40374
I20260925 10:57:40.912463 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 13501
I20260925 10:57:40.919703 13939 tablet_replica.cc:330] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685: stopping tablet replica
I20260925 10:57:40.920277 13939 raft_consensus.cc:2229] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:57:40.921044 13939 raft_consensus.cc:2258] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:57:40.923411 13939 ts_tablet_manager.cc:1905] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:57:40.926297 13938 tablet_replica.cc:330] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778: stopping tablet replica
I20260925 10:57:40.926476 13940 tablet_replica.cc:330] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899: stopping tablet replica
I20260925 10:57:40.926786 13938 raft_consensus.cc:2229] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:57:40.926885 13940 raft_consensus.cc:2229] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:57:40.927394 13940 raft_consensus.cc:2258] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:57:40.927416 13938 raft_consensus.cc:2258] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:57:40.930020 13940 ts_tablet_manager.cc:1905] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:57:40.931329 13939 ts_tablet_manager.cc:1918] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:57:40.931564 13939 log.cc:1196] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/53ea2d47923e47c69285637659f8aa16
Sep 25 10:57:40 dist-test-slave-nx7s krb5kdc[13395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:57:40.932147 13939 ts_tablet_manager.cc:1939] T 53ea2d47923e47c69285637659f8aa16 P 7a4510b7224f42049881f3d65bd68685: Deleting consensus metadata
I20260925 10:57:40.933391 13938 ts_tablet_manager.cc:1905] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:57:40.934152 13425 catalog_manager.cc:4841] TS 7a4510b7224f42049881f3d65bd68685 (127.31.62.130:32815): tablet 53ea2d47923e47c69285637659f8aa16 (table test-table [id=3270f1c98e44466aa8ccdebf422458b9]) successfully deleted
I20260925 10:57:40.937283 13940 ts_tablet_manager.cc:1918] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:57:40.937556 13940 log.cc:1196] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/53ea2d47923e47c69285637659f8aa16
I20260925 10:57:40.938169 13940 ts_tablet_manager.cc:1939] T 53ea2d47923e47c69285637659f8aa16 P 38921275dbb34d378e099ae3b33fe899: Deleting consensus metadata
I20260925 10:57:40.940245 13426 catalog_manager.cc:4841] TS 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007): tablet 53ea2d47923e47c69285637659f8aa16 (table test-table [id=3270f1c98e44466aa8ccdebf422458b9]) successfully deleted
I20260925 10:57:40.940696 13938 ts_tablet_manager.cc:1918] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:57:40.940900 13938 log.cc:1196] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/53ea2d47923e47c69285637659f8aa16
I20260925 10:57:40.941366 13938 ts_tablet_manager.cc:1939] T 53ea2d47923e47c69285637659f8aa16 P 5423854314a740c5a7f9d497df7e9778: Deleting consensus metadata
I20260925 10:57:40.943127 13426 catalog_manager.cc:4841] TS 5423854314a740c5a7f9d497df7e9778 (127.31.62.129:34229): tablet 53ea2d47923e47c69285637659f8aa16 (table test-table [id=3270f1c98e44466aa8ccdebf422458b9]) successfully deleted
I20260925 10:57:40.944062 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:40.913814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41836 (local address 127.31.62.129:34229)
0925 10:57:40.914084 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:40.914091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:40.920051 (+ 5960us) server_negotiation.cc:407] Connection header received
0925 10:57:40.920341 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:40.920347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:40.920438 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:40.920518 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:40.922818 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.924098 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.926358 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:40.927043 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:40.927172 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:40.935980 (+ 8808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:40.935998 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:40.936017 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:40.936066 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:40.940405 (+ 4339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:40.941613 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:40.941624 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:40.941627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:40.941807 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:40.942363 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:40.942367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:40.942370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:40.942855 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:40.943144 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:40.943506 (+ 362us) server_negotiation.cc:299] Negotiation successful
0925 10:57:40.943731 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 10:57:40.944759 13624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:41.035454 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 13501
I20260925 10:57:41.050422 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 13635
Sep 25 10:57:41 dist-test-slave-nx7s krb5kdc[13395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:57:41.068289 13911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:41.051823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45718 (local address 127.31.62.130:32815)
0925 10:57:41.052095 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:41.052101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:41.052122 (+ 21us) server_negotiation.cc:407] Connection header received
0925 10:57:41.052189 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:41.052195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:41.052281 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:41.052390 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:41.053664 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:41.054870 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:41.056167 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:41.056927 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:41.057056 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:41.061431 (+ 4375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:41.061456 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:41.061478 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:41.061535 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:41.065484 (+ 3949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:41.066160 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:41.066171 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:41.066174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:41.066350 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:41.066677 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:41.066682 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:41.066686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:41.067161 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:41.067490 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:41.067659 (+ 169us) server_negotiation.cc:299] Negotiation successful
0925 10:57:41.067889 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260925 10:57:41.069154 13758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:41.179137 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 13635
I20260925 10:57:41.193673 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 13768
Sep 25 10:57:41 dist-test-slave-nx7s krb5kdc[13395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:57:41.208436 13913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:41.194408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40390 (local address 127.31.62.131:42007)
0925 10:57:41.194566 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:41.194570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:41.194626 (+ 56us) server_negotiation.cc:407] Connection header received
0925 10:57:41.194771 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:41.194776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:41.194846 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:41.194929 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:41.196200 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:41.196962 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:41.198193 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:41.198577 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:41.198667 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:41.203130 (+ 4463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:41.203146 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:41.203162 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:41.203198 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:41.205918 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:41.206537 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:41.206545 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:41.206547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:41.206693 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:41.207123 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:41.207127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:41.207128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:41.207481 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:41.207703 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:41.208004 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 10:57:41.208188 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 10:57:41.209153 13891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:41.297115 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 13768
I20260925 10:57:41.310220 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 13408
I20260925 10:57:41.311328 13470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:41.426234 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 13408
I20260925 10:57:41.443266 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41871
--webserver_interface=127.31.62.190
--webserver_port=44107
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:41871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:57:41.768900 13946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:41.769279 13946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:41.769436 13946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:41.776782 13946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:57:41.776949 13946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:41.777055 13946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:41.777119 13946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:57:41.777179 13946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:57:41.787441 13946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:41871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=44107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:41.788784 13946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:41.790113 13946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:41.811173 13952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:41.811205 13953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:41.811095 13955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:41.811839 13946 server_base.cc:1034] running on GCE node
I20260925 10:57:41.812690 13946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:41.814342 13946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:41.815563 13946 hybrid_clock.cc:648] HybridClock initialized: now 1790333861815480 us; error 66 us; skew 500 ppm
I20260925 10:57:41.816128 13946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:41 dist-test-slave-nx7s krb5kdc[13395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333861, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:41.819607 13946 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:57:41.821344 13946 webserver.cc:460] Webserver started at http://127.31.62.190:44107/ using document root <none> and password file <none>
I20260925 10:57:41.822098 13946 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:41.822219 13946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:41.828588 13946 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260925 10:57:41.831674 13962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:41.833285 13946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:57:41.833459 13946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "0e68e89e7e734035b161024a74256219"
format_stamp: "Formatted at 2026-09-25 10:57:38 on dist-test-slave-nx7s"
server_key: "9fd4bda4585a30c38d63d86c082ae080"
server_key_iv: "ced6b6c69ba096503111472cb2974fb6"
server_key_version: "encryptionkey@0"
I20260925 10:57:41.834623 13946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:41.847886 13946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:41.852469 13946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:41.852749 13946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:41.870748 13946 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:41871
I20260925 10:57:41.870772 14014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:41871 every 8 connection(s)
I20260925 10:57:41.872283 13946 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:57:41.874884 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 13946
I20260925 10:57:41.875833 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:34229
--local_ip_for_outbound_sockets=127.31.62.129
--tserver_master_addrs=127.31.62.190:41871
--webserver_port=44759
--webserver_interface=127.31.62.129
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:57:41.881464 14015 sys_catalog.cc:263] Verifying existing consensus state
I20260925 10:57:41.885613 14015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: Bootstrap starting.
I20260925 10:57:41.905490 14015 log.cc:824] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:41.917392 14015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 10:57:41.917898 14015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: Bootstrap complete.
I20260925 10:57:41.927174 14015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } }
I20260925 10:57:41.928133 14015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e68e89e7e734035b161024a74256219, State: Initialized, Role: FOLLOWER
I20260925 10:57:41.928763 14015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } }
I20260925 10:57:41.929040 14015 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:57:41.929150 14015 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:57:41.929317 14015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 1 FOLLOWER]: Advancing to term 2
I20260925 10:57:41.933313 14015 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } }
I20260925 10:57:41.933871 14015 leader_election.cc:302] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e68e89e7e734035b161024a74256219; no voters:
I20260925 10:57:41.934486 14015 leader_election.cc:288] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 10:57:41.934618 14019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 10:57:41.935670 14019 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 2 LEADER]: Becoming Leader. State: Replica: 0e68e89e7e734035b161024a74256219, State: Running, Role: LEADER
I20260925 10:57:41.936297 14019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } }
I20260925 10:57:41.936760 14015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:57:41.938848 14021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e68e89e7e734035b161024a74256219. Latest consensus state: current_term: 2 leader_uuid: "0e68e89e7e734035b161024a74256219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } } }
I20260925 10:57:41.939198 14021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [sys.catalog]: This master's current role is: LEADER
I20260925 10:57:41.939654 14020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0e68e89e7e734035b161024a74256219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } } }
I20260925 10:57:41.939973 14020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [sys.catalog]: This master's current role is: LEADER
I20260925 10:57:41.940163 14024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:57:41.960104 14024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:57:41.961845 14024 catalog_manager.cc:1259] Loaded cluster ID: 2be48f33445145cea72e0353d86b563c
I20260925 10:57:41.962062 14024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:57:41.965840 14024 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260925 10:57:41.984068 14024 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260925 10:57:42.006052 14024 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:57:42.008169 14024 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: Loaded TSK: 0
I20260925 10:57:42.010479 14024 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 10:57:42.250072 14017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:42.250383 14017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:42.250519 14017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:42.257946 14017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:42.258085 14017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:42.258234 14017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:57:42.268368 14017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:34229
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=44759
--tserver_master_addrs=127.31.62.190:41871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:42.269778 14017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:42.271142 14017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:42.286466 14043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:42.286753 14045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:42.286485 14042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:42.287801 14017 server_base.cc:1034] running on GCE node
I20260925 10:57:42.288461 14017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:42.289678 14017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:42.290915 14017 hybrid_clock.cc:648] HybridClock initialized: now 1790333862290829 us; error 75 us; skew 500 ppm
I20260925 10:57:42.291283 14017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:42 dist-test-slave-nx7s krb5kdc[13395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333862, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:42.294806 14017 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:57:42.296469 14017 webserver.cc:460] Webserver started at http://127.31.62.129:44759/ using document root <none> and password file <none>
I20260925 10:57:42.297253 14017 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:42.297386 14017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:42.303273 14017 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260925 10:57:42.306603 14052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:42.308210 14017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:57:42.308409 14017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "5423854314a740c5a7f9d497df7e9778"
format_stamp: "Formatted at 2026-09-25 10:57:39 on dist-test-slave-nx7s"
server_key: "a2d1be3111f43a29333c2698ac8d6ee5"
server_key_iv: "7056ee984f8c4dcf4843ef67fee40a7e"
server_key_version: "encryptionkey@0"
I20260925 10:57:42.309572 14017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:42.330165 14017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:42.334656 14017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:42.334914 14017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:42.335955 14017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:57:42.337455 14017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:57:42.337570 14017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:42.337721 14017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:57:42.337785 14017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:42.369006 14017 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:34229
I20260925 10:57:42.369059 14165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:34229 every 8 connection(s)
I20260925 10:57:42.370539 14017 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:57:42.380884 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 14017
I20260925 10:57:42.381500 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:32815
--local_ip_for_outbound_sockets=127.31.62.130
--tserver_master_addrs=127.31.62.190:41871
--webserver_port=43865
--webserver_interface=127.31.62.130
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 10:57:42 dist-test-slave-nx7s krb5kdc[13395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333862, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:42.395272 14169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:42.373805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:47349)
0925 10:57:42.374620 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:42.374665 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:42.376470 (+ 1805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:42.378067 (+ 1597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:42.378078 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:42.378686 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:42.379249 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:42.379268 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:42.380912 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:42.380916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:42.382263 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:42.382276 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:42.383376 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:42.383383 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:42.383655 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:42.384304 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:42.384336 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:42.386972 (+ 2636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:42.391092 (+ 4120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:42.391100 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:42.391114 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:42.391581 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:42.392085 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:42.392088 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:42.392090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:42.392273 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:42.393209 (+ 936us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:42.393216 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:42.393554 (+ 338us) client_negotiation.cc:769] Sending connection context
0925 10:57:42.393785 (+ 231us) client_negotiation.cc:240] Negotiation successful
0925 10:57:42.394236 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":348,"threads_started":1}
I20260925 10:57:42.395550 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:42.373425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:47349 (local address 127.31.62.190:41871)
0925 10:57:42.374010 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:42.374022 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:42.374807 (+ 785us) server_negotiation.cc:407] Connection header received
0925 10:57:42.377122 (+ 2315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:42.377148 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:42.377524 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:42.377879 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:42.379471 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:42.380721 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:42.382409 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:42.383136 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:42.383312 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:42.387296 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:42.387305 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:42.387321 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:42.387362 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:42.390910 (+ 3548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:42.391724 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:42.391733 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:42.391741 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:42.391962 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:42.392372 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:42.392376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:42.392379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:42.393074 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:42.393257 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:42.394027 (+ 770us) server_negotiation.cc:299] Negotiation successful
0925 10:57:42.394449 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":252,"threads_started":1}
I20260925 10:57:42.398693 14166 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41871
I20260925 10:57:42.399050 14166 heartbeater.cc:464] Registering TS with master...
I20260925 10:57:42.399996 14166 heartbeater.cc:511] Master 127.31.62.190:41871 requested a full tablet report, sending...
I20260925 10:57:42.402614 13979 ts_manager.cc:194] Registered new tserver with Master: 5423854314a740c5a7f9d497df7e9778 (127.31.62.129:34229)
I20260925 10:57:42.404772 13979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:47349
W20260925 10:57:42.705319 14170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:42.705623 14170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:42.705721 14170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:42.712975 14170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:42.713111 14170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:42.713230 14170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:57:42.723538 14170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:32815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=43865
--tserver_master_addrs=127.31.62.190:41871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:42.724824 14170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:42.726132 14170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:42.738843 14175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:42.738809 14176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:42.738796 14178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:42.740284 14170 server_base.cc:1034] running on GCE node
I20260925 10:57:42.740929 14170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:42.741892 14170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:42.743130 14170 hybrid_clock.cc:648] HybridClock initialized: now 1790333862743073 us; error 56 us; skew 500 ppm
I20260925 10:57:42.743427 14170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:42 dist-test-slave-nx7s krb5kdc[13395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333862, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:42.746461 14170 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:57:42.747903 14170 webserver.cc:460] Webserver started at http://127.31.62.130:43865/ using document root <none> and password file <none>
I20260925 10:57:42.748572 14170 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:42.748692 14170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:42.754076 14170 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:57:42.756898 14185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:42.758263 14170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 10:57:42.758424 14170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "7a4510b7224f42049881f3d65bd68685"
format_stamp: "Formatted at 2026-09-25 10:57:40 on dist-test-slave-nx7s"
server_key: "71214e07852e7309d7bd50b9182185fd"
server_key_iv: "ca54fa1bceaa23f1a7b9d8992ea9f039"
server_key_version: "encryptionkey@0"
I20260925 10:57:42.759407 14170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:42.780593 14170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:42.785547 14170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:42.785848 14170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:42.786862 14170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:57:42.788571 14170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:57:42.788684 14170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:42.788825 14170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:57:42.788939 14170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260925 10:57:42.820247 14170 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:32815
I20260925 10:57:42.820302 14298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:32815 every 8 connection(s)
I20260925 10:57:42.821758 14170 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:57:42.825700 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 14170
I20260925 10:57:42.826526 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:42007
--local_ip_for_outbound_sockets=127.31.62.131
--tserver_master_addrs=127.31.62.190:41871
--webserver_port=41849
--webserver_interface=127.31.62.131
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 10:57:42 dist-test-slave-nx7s krb5kdc[13395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333862, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:42.847218 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:42.824386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:56935 (local address 127.31.62.190:41871)
0925 10:57:42.824563 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:42.824568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:42.826124 (+ 1556us) server_negotiation.cc:407] Connection header received
0925 10:57:42.828901 (+ 2777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:42.828908 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:42.829000 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:42.829105 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:42.831375 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:42.832306 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:42.834735 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:42.835387 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:42.835537 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:42.840001 (+ 4464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:42.840009 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:42.840014 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:42.840058 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:42.842703 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:42.843681 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:42.843686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:42.843688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:42.843825 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:42.844808 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:42.844814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:42.844818 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:42.845255 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:42.845418 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:42.846582 (+ 1164us) server_negotiation.cc:299] Negotiation successful
0925 10:57:42.846853 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 10:57:42.848462 14301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:42.824798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:56935)
0925 10:57:42.825789 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:42.825933 (+ 144us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:42.828503 (+ 2570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:42.829438 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:42.829452 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:42.830223 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:42.831152 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:42.831183 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:42.832493 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:42.832498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:42.834550 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:42.834566 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:42.835660 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:42.835666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:42.835963 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:42.836762 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:42.836799 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:42.839745 (+ 2946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:42.842871 (+ 3126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:42.842883 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:42.842903 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:42.843529 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:42.844235 (+ 706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:42.844260 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:42.844263 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:42.844560 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:42.845497 (+ 937us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:42.845543 (+ 46us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:42.846073 (+ 530us) client_negotiation.cc:769] Sending connection context
0925 10:57:42.846451 (+ 378us) client_negotiation.cc:240] Negotiation successful
0925 10:57:42.847036 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":344,"threads_started":1}
I20260925 10:57:42.850219 14299 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41871
I20260925 10:57:42.850598 14299 heartbeater.cc:464] Registering TS with master...
I20260925 10:57:42.851622 14299 heartbeater.cc:511] Master 127.31.62.190:41871 requested a full tablet report, sending...
I20260925 10:57:42.853384 13979 ts_manager.cc:194] Registered new tserver with Master: 7a4510b7224f42049881f3d65bd68685 (127.31.62.130:32815)
I20260925 10:57:42.854310 13979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:56935
W20260925 10:57:43.167876 14302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:43.168224 14302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:43.168386 14302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:43.178694 14302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:43.178812 14302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:43.179013 14302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:57:43.194715 14302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:42007
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=41849
--tserver_master_addrs=127.31.62.190:41871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:43.196560 14302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:43.198458 14302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:43.212463 14308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:43.212586 14310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:43.212751 14307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:43.213793 14302 server_base.cc:1034] running on GCE node
I20260925 10:57:43.214514 14302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:43.215519 14302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:43.216740 14302 hybrid_clock.cc:648] HybridClock initialized: now 1790333863216687 us; error 45 us; skew 500 ppm
I20260925 10:57:43.217104 14302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:43 dist-test-slave-nx7s krb5kdc[13395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333863, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:43.220497 14302 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:57:43.222229 14302 webserver.cc:460] Webserver started at http://127.31.62.131:41849/ using document root <none> and password file <none>
I20260925 10:57:43.222936 14302 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:43.223084 14302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:43.228789 14302 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 10:57:43.231854 14317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:43.233379 14302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:57:43.233546 14302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "38921275dbb34d378e099ae3b33fe899"
format_stamp: "Formatted at 2026-09-25 10:57:40 on dist-test-slave-nx7s"
server_key: "c05ceadf40192d675f121e371e0ab0b1"
server_key_iv: "77c5ed51922dba831d0fe306a154b304"
server_key_version: "encryptionkey@0"
I20260925 10:57:43.234635 14302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:43.246800 14302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:43.251448 14302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:43.251701 14302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:43.252717 14302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:57:43.254153 14302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:57:43.254237 14302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:43.254335 14302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:57:43.254400 14302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:43.284581 14302 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:42007
I20260925 10:57:43.284698 14430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:42007 every 8 connection(s)
I20260925 10:57:43.286131 14302 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:57:43.292235 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 14302
Sep 25 10:57:43 dist-test-slave-nx7s krb5kdc[13395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333863, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:57:43.306905 14435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.293253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45010 (local address 127.31.62.190:41871)
0925 10:57:43.294049 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:43.294055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:43.294071 (+ 16us) server_negotiation.cc:407] Connection header received
0925 10:57:43.294120 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:43.294124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:43.294177 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:43.294265 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:43.295759 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.296522 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.297904 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.298523 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.298633 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.301365 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:43.301370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:43.301373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:43.301419 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:43.304150 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:43.304842 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:43.304866 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:43.304869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:43.305034 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:43.305396 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:43.305401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:43.305405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:43.305719 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:43.305958 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:43.306285 (+ 327us) server_negotiation.cc:299] Negotiation successful
0925 10:57:43.306509 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":548,"threads_started":1}
I20260925 10:57:43.307267 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.288781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:39137 (local address 127.31.62.190:41871)
0925 10:57:43.289003 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:43.289009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:43.290310 (+ 1301us) server_negotiation.cc:407] Connection header received
0925 10:57:43.292188 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:43.292194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:43.292289 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:43.292407 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:43.294424 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.295154 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.296833 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.297295 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.297411 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.301198 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:43.301203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:43.301206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:43.301240 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:43.304071 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:43.304877 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:43.304882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:43.304885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:43.305047 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:43.305483 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:43.305487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:43.305489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:43.305761 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:43.305887 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:43.306630 (+ 743us) server_negotiation.cc:299] Negotiation successful
0925 10:57:43.306839 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"spinlock_wait_cycles":5888}
I20260925 10:57:43.308482 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.289406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:39137)
0925 10:57:43.290136 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:43.290180 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:43.291889 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:43.292672 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:43.292681 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:43.293374 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:43.294238 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:43.294260 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.295342 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.295346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:43.296681 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:43.296695 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.297567 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.297574 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:43.297818 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.298551 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:43.298575 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:43.300965 (+ 2390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:43.304226 (+ 3261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:43.304236 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:43.304256 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:43.304728 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:43.305161 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:43.305166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:43.305170 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:43.305362 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:43.305891 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:43.305901 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:43.306263 (+ 362us) client_negotiation.cc:769] Sending connection context
0925 10:57:43.306511 (+ 248us) client_negotiation.cc:240] Negotiation successful
0925 10:57:43.307103 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":278,"threads_started":1}
I20260925 10:57:43.309911 14431 heartbeater.cc:347] Connected to a master server at 127.31.62.190:41871
I20260925 10:57:43.310204 14431 heartbeater.cc:464] Registering TS with master...
I20260925 10:57:43.310911 14431 heartbeater.cc:511] Master 127.31.62.190:41871 requested a full tablet report, sending...
I20260925 10:57:43.312330 13978 ts_manager.cc:194] Registered new tserver with Master: 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007)
I20260925 10:57:43.313306 13978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:39137
I20260925 10:57:43.322938 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:57:43.330564 14435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.323935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45024 (local address 127.31.62.190:41871)
0925 10:57:43.324119 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:43.324123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:43.324532 (+ 409us) server_negotiation.cc:407] Connection header received
0925 10:57:43.324770 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:43.324775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:43.324989 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:43.325145 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:57:43.325549 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.326327 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.328172 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.328600 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.328708 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.329444 (+ 736us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:57:43.330014 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:57:43.330161 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:43.330276 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 10:57:43.330354 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 10:57:43.331573 13978 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:45024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:57:43.339460 13978 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:57:43.366380 14441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.352103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:42007 (local address 127.0.0.1:40392)
0925 10:57:43.354678 (+ 2575us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:43.354736 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:43.354928 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:43.357143 (+ 2215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:43.357147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:43.357172 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:57:43.357519 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:43.357531 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.360519 (+ 2988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.360524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:43.362759 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:43.362781 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.365283 (+ 2502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.365289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:43.365502 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.365560 (+ 58us) client_negotiation.cc:769] Sending connection context
0925 10:57:43.365702 (+ 142us) client_negotiation.cc:240] Negotiation successful
0925 10:57:43.365902 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2351,"mutex_wait_us":3,"thread_start_us":2055,"threads_started":1}
I20260925 10:57:43.366993 14444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.352495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40392 (local address 127.31.62.131:42007)
0925 10:57:43.353949 (+ 1454us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:43.353957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:43.356217 (+ 2260us) server_negotiation.cc:407] Connection header received
0925 10:57:43.356417 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:43.356426 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:43.356678 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:43.356879 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:57:43.357711 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.360360 (+ 2649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.363547 (+ 3187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.364738 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.364825 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.365092 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:43.366495 (+ 1403us) server_negotiation.cc:299] Negotiation successful
0925 10:57:43.366675 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1287,"thread_start_us":1139,"threads_started":1}
I20260925 10:57:43.368721 14443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.352396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45726 (local address 127.31.62.130:32815)
0925 10:57:43.353457 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:43.353467 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:43.354871 (+ 1404us) server_negotiation.cc:407] Connection header received
0925 10:57:43.355267 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:43.355278 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:43.355713 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:43.355985 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:57:43.357411 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.359460 (+ 2049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.365280 (+ 5820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.366751 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.366892 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.367101 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:43.368180 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0925 10:57:43.368393 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":881,"thread_start_us":756,"threads_started":1}
I20260925 10:57:43.371141 14366 tablet_service.cc:1467] Processing CreateTablet for tablet d51fb76e7f0f4e37b2fcb225d82d673e (DEFAULT_TABLE table=test-table [id=9d079940df4a4c4b80d8c22c9b2cea52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:57:43.372418 14366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d51fb76e7f0f4e37b2fcb225d82d673e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:57:43.373365 14442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.351972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:32815 (local address 127.0.0.1:45726)
0925 10:57:43.354678 (+ 2706us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:43.354762 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:43.354975 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:43.356420 (+ 1445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:43.356428 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:43.356483 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:57:43.357094 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:43.357135 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.359718 (+ 2583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.359725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:43.365094 (+ 5369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:43.365109 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.366971 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.366978 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:43.367968 (+ 990us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.368002 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 10:57:43.372742 (+ 4740us) client_negotiation.cc:240] Negotiation successful
0925 10:57:43.372925 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2387,"mutex_wait_us":2324,"thread_start_us":3980,"threads_started":1}
I20260925 10:57:43.374328 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.355779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41846 (local address 127.31.62.129:34229)
0925 10:57:43.365291 (+ 9512us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:43.365300 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:43.365326 (+ 26us) server_negotiation.cc:407] Connection header received
0925 10:57:43.365419 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:43.365425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:43.365635 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:43.365821 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:57:43.366373 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.368075 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.370785 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.372186 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.372323 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.372536 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:43.373798 (+ 1262us) server_negotiation.cc:299] Negotiation successful
0925 10:57:43.373977 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9347,"thread_start_us":9231,"threads_started":1}
I20260925 10:57:43.374611 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.355142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34229 (local address 127.0.0.1:41846)
0925 10:57:43.363251 (+ 8109us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:43.363270 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:43.363375 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:43.365824 (+ 2449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:43.365829 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:43.365854 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:57:43.366204 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:43.366216 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.368235 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.368240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:43.370604 (+ 2364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:43.370621 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.372672 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.372679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:43.372878 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.372924 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 10:57:43.373019 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 10:57:43.373151 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7971,"thread_start_us":7799,"threads_started":1}
I20260925 10:57:43.377513 14234 tablet_service.cc:1467] Processing CreateTablet for tablet d51fb76e7f0f4e37b2fcb225d82d673e (DEFAULT_TABLE table=test-table [id=9d079940df4a4c4b80d8c22c9b2cea52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:57:43.378966 14234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d51fb76e7f0f4e37b2fcb225d82d673e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:57:43.378464 14101 tablet_service.cc:1467] Processing CreateTablet for tablet d51fb76e7f0f4e37b2fcb225d82d673e (DEFAULT_TABLE table=test-table [id=9d079940df4a4c4b80d8c22c9b2cea52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:57:43.379817 14101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d51fb76e7f0f4e37b2fcb225d82d673e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:57:43.386616 14447 tablet_bootstrap.cc:492] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899: Bootstrap starting.
I20260925 10:57:43.389599 14447 tablet_bootstrap.cc:654] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899: Neither blocks nor log segments found. Creating new log.
I20260925 10:57:43.390612 14447 log.cc:824] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:43.393157 14447 tablet_bootstrap.cc:492] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899: No bootstrap required, opened a new log
I20260925 10:57:43.393435 14447 ts_tablet_manager.cc:1397] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260925 10:57:43.395610 14449 tablet_bootstrap.cc:492] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685: Bootstrap starting.
I20260925 10:57:43.397266 14450 tablet_bootstrap.cc:492] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778: Bootstrap starting.
I20260925 10:57:43.399466 14449 tablet_bootstrap.cc:654] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685: Neither blocks nor log segments found. Creating new log.
I20260925 10:57:43.399998 14450 tablet_bootstrap.cc:654] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778: Neither blocks nor log segments found. Creating new log.
I20260925 10:57:43.400738 14449 log.cc:824] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:43.400983 14450 log.cc:824] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:43.403252 14447 raft_consensus.cc:348] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:57:43.403635 14447 raft_consensus.cc:374] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:57:43.403728 14447 raft_consensus.cc:729] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38921275dbb34d378e099ae3b33fe899, State: Initialized, Role: FOLLOWER
I20260925 10:57:43.403895 14449 tablet_bootstrap.cc:492] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685: No bootstrap required, opened a new log
I20260925 10:57:43.403894 14450 tablet_bootstrap.cc:492] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778: No bootstrap required, opened a new log
I20260925 10:57:43.404193 14450 ts_tablet_manager.cc:1397] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260925 10:57:43.404193 14449 ts_tablet_manager.cc:1397] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260925 10:57:43.404384 14447 consensus_queue.cc:260] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:57:43.407678 14431 heartbeater.cc:503] Master 127.31.62.190:41871 was elected leader, sending a full tablet report...
I20260925 10:57:43.408423 14447 ts_tablet_manager.cc:1428] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260925 10:57:43.408962 14166 heartbeater.cc:503] Master 127.31.62.190:41871 was elected leader, sending a full tablet report...
I20260925 10:57:43.414824 14449 raft_consensus.cc:348] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:57:43.414824 14450 raft_consensus.cc:348] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:57:43.415302 14450 raft_consensus.cc:374] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:57:43.415302 14449 raft_consensus.cc:374] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:57:43.415407 14449 raft_consensus.cc:729] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a4510b7224f42049881f3d65bd68685, State: Initialized, Role: FOLLOWER
I20260925 10:57:43.415408 14450 raft_consensus.cc:729] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5423854314a740c5a7f9d497df7e9778, State: Initialized, Role: FOLLOWER
I20260925 10:57:43.416064 14449 consensus_queue.cc:260] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:57:43.416066 14450 consensus_queue.cc:260] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:57:43.417250 14299 heartbeater.cc:503] Master 127.31.62.190:41871 was elected leader, sending a full tablet report...
I20260925 10:57:43.418057 14449 ts_tablet_manager.cc:1428] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260925 10:57:43.418097 14450 ts_tablet_manager.cc:1428] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
W20260925 10:57:43.537703 14432 tablet.cc:2369] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:57:43.573655 14300 tablet.cc:2369] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:57:43.586833 14455 raft_consensus.cc:482] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:57:43.587165 14455 raft_consensus.cc:504] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:57:43.588874 14455 leader_election.cc:288] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a4510b7224f42049881f3d65bd68685 (127.31.62.130:32815), 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007)
I20260925 10:57:43.595818 14444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.589302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:50971 (local address 127.31.62.131:42007)
0925 10:57:43.589502 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:43.589509 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:43.590287 (+ 778us) server_negotiation.cc:407] Connection header received
0925 10:57:43.590474 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:43.590478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:43.590545 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:43.590616 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:57:43.591228 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.591942 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.594034 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.594834 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.594991 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.595071 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:43.595366 (+ 295us) server_negotiation.cc:299] Negotiation successful
0925 10:57:43.595529 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 10:57:43.595920 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.589308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:42007 (local address 127.31.62.129:50971)
0925 10:57:43.590178 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:43.590193 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:43.590343 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:43.590644 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:43.590648 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:43.590670 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:57:43.591095 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:43.591105 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.592070 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.592075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:43.593881 (+ 1806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:43.593893 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.595045 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.595051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:43.595213 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.595239 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 10:57:43.595319 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 10:57:43.595458 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":68,"thread_start_us":438,"threads_started":1}
I20260925 10:57:43.595924 14443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.589318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:38129 (local address 127.31.62.130:32815)
0925 10:57:43.589512 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:43.589517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:43.590328 (+ 811us) server_negotiation.cc:407] Connection header received
0925 10:57:43.590491 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:43.590496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:43.590558 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:43.590639 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:57:43.591268 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.591959 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.594224 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.594998 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.595160 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.595236 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:43.595536 (+ 300us) server_negotiation.cc:299] Negotiation successful
0925 10:57:43.595676 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 10:57:43.596076 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.589339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:32815 (local address 127.31.62.129:38129)
0925 10:57:43.590226 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:43.590241 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:43.590365 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:43.590663 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:43.590668 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:43.590690 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:57:43.591100 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:43.591108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.592085 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.592091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:43.594064 (+ 1973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:43.594076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.595214 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.595220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:43.595375 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.595398 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 10:57:43.595498 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 10:57:43.595642 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"mutex_wait_us":43,"thread_start_us":419,"threads_started":1}
I20260925 10:57:43.597018 14254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d51fb76e7f0f4e37b2fcb225d82d673e" candidate_uuid: "5423854314a740c5a7f9d497df7e9778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a4510b7224f42049881f3d65bd68685" is_pre_election: true
I20260925 10:57:43.597030 14386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d51fb76e7f0f4e37b2fcb225d82d673e" candidate_uuid: "5423854314a740c5a7f9d497df7e9778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38921275dbb34d378e099ae3b33fe899" is_pre_election: true
I20260925 10:57:43.597486 14254 raft_consensus.cc:2454] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5423854314a740c5a7f9d497df7e9778 in term 0.
I20260925 10:57:43.597479 14386 raft_consensus.cc:2454] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5423854314a740c5a7f9d497df7e9778 in term 0.
I20260925 10:57:43.598093 14054 leader_election.cc:302] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5423854314a740c5a7f9d497df7e9778, 7a4510b7224f42049881f3d65bd68685; no voters:
I20260925 10:57:43.598457 14455 raft_consensus.cc:2781] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:57:43.598608 14455 raft_consensus.cc:482] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:57:43.598707 14455 raft_consensus.cc:3037] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:57:43.600814 14455 raft_consensus.cc:504] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:57:43.601450 14455 leader_election.cc:288] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 1 election: Requested vote from peers 7a4510b7224f42049881f3d65bd68685 (127.31.62.130:32815), 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007)
I20260925 10:57:43.601899 14254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d51fb76e7f0f4e37b2fcb225d82d673e" candidate_uuid: "5423854314a740c5a7f9d497df7e9778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a4510b7224f42049881f3d65bd68685"
I20260925 10:57:43.601920 14386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d51fb76e7f0f4e37b2fcb225d82d673e" candidate_uuid: "5423854314a740c5a7f9d497df7e9778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38921275dbb34d378e099ae3b33fe899"
I20260925 10:57:43.602097 14386 raft_consensus.cc:3037] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:57:43.602092 14254 raft_consensus.cc:3037] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:57:43.604205 14386 raft_consensus.cc:2454] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5423854314a740c5a7f9d497df7e9778 in term 1.
I20260925 10:57:43.604231 14254 raft_consensus.cc:2454] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5423854314a740c5a7f9d497df7e9778 in term 1.
I20260925 10:57:43.604682 14055 leader_election.cc:302] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38921275dbb34d378e099ae3b33fe899, 5423854314a740c5a7f9d497df7e9778; no voters:
I20260925 10:57:43.605062 14455 raft_consensus.cc:2781] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:57:43.605504 14455 raft_consensus.cc:686] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 LEADER]: Becoming Leader. State: Replica: 5423854314a740c5a7f9d497df7e9778, State: Running, Role: LEADER
I20260925 10:57:43.606027 14455 consensus_queue.cc:237] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:57:43.610512 13978 catalog_manager.cc:5495] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5423854314a740c5a7f9d497df7e9778 (127.31.62.129). New cstate: current_term: 1 leader_uuid: "5423854314a740c5a7f9d497df7e9778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } health_report { overall_health: HEALTHY } } }
W20260925 10:57:43.622822 14167 tablet.cc:2369] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:57:43.662910 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.657517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41848 (local address 127.31.62.129:34229)
0925 10:57:43.657724 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:43.657729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:43.657749 (+ 20us) server_negotiation.cc:407] Connection header received
0925 10:57:43.657972 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:43.657976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:43.658055 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:43.658188 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:57:43.658565 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.659288 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.660641 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.661092 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.661201 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.661723 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:57:43.662296 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:57:43.662445 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:43.662546 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 10:57:43.662635 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 10:57:43.671100 14386 raft_consensus.cc:1261] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899 [term 1 FOLLOWER]: Refusing update from remote peer 5423854314a740c5a7f9d497df7e9778: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:57:43.671106 14254 raft_consensus.cc:1261] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685 [term 1 FOLLOWER]: Refusing update from remote peer 5423854314a740c5a7f9d497df7e9778: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:57:43.672066 14458 consensus_queue.cc:1035] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:57:43.672417 14455 consensus_queue.cc:1035] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:57:43.684935 14463 mvcc.cc:205] Tried to move back new op lower bound from 7333207505587011584 to 7333207505334071296. Current Snapshot: MvccSnapshot[applied={T|T < 7333207505587011584}]
I20260925 10:57:43.700465 13977 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:45024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:57:43.700811 13977 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:45024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:57:43.704802 13977 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: Sending DeleteTablet for 3 replicas of tablet d51fb76e7f0f4e37b2fcb225d82d673e
I20260925 10:57:43.706018 14234 tablet_service.cc:1514] Processing DeleteTablet for tablet d51fb76e7f0f4e37b2fcb225d82d673e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:57:43 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:45726
I20260925 10:57:43.706103 14366 tablet_service.cc:1514] Processing DeleteTablet for tablet d51fb76e7f0f4e37b2fcb225d82d673e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:57:43 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:40392
I20260925 10:57:43.706249 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 14017
I20260925 10:57:43.706811 14101 tablet_service.cc:1514] Processing DeleteTablet for tablet d51fb76e7f0f4e37b2fcb225d82d673e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:57:43 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:41846
I20260925 10:57:43.722441 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.707015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41860 (local address 127.31.62.129:34229)
0925 10:57:43.707178 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:43.707184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:43.707318 (+ 134us) server_negotiation.cc:407] Connection header received
0925 10:57:43.707495 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:43.707501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:43.707586 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:43.707667 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:43.709370 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.710141 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.712061 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.712484 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.712600 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.715990 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:43.716001 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:43.716018 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:43.716058 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:43.719479 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:43.720196 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:43.720204 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:43.720207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:43.720377 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:43.720730 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:43.720735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:43.720738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:43.721239 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:43.721597 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:43.721846 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 10:57:43.722075 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 10:57:43.723301 14161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:43.724277 14471 tablet_replica.cc:330] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899: stopping tablet replica
I20260925 10:57:43.724840 14471 raft_consensus.cc:2229] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:57:43.725505 14471 raft_consensus.cc:2258] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:57:43.727986 14472 tablet_replica.cc:330] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685: stopping tablet replica
I20260925 10:57:43.728117 14471 ts_tablet_manager.cc:1905] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:57:43.728542 14472 raft_consensus.cc:2229] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:57:43.729084 14472 raft_consensus.cc:2258] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:57:43.731218 14472 ts_tablet_manager.cc:1905] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:57:43.733271 14471 ts_tablet_manager.cc:1918] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:57:43.733409 14471 log.cc:1196] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/d51fb76e7f0f4e37b2fcb225d82d673e
I20260925 10:57:43.733815 14471 ts_tablet_manager.cc:1939] T d51fb76e7f0f4e37b2fcb225d82d673e P 38921275dbb34d378e099ae3b33fe899: Deleting consensus metadata
I20260925 10:57:43.735441 13965 catalog_manager.cc:4841] TS 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007): tablet d51fb76e7f0f4e37b2fcb225d82d673e (table test-table [id=9d079940df4a4c4b80d8c22c9b2cea52]) successfully deleted
I20260925 10:57:43.738267 14472 ts_tablet_manager.cc:1918] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:57:43.738430 14472 log.cc:1196] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/d51fb76e7f0f4e37b2fcb225d82d673e
I20260925 10:57:43.738996 14472 ts_tablet_manager.cc:1939] T d51fb76e7f0f4e37b2fcb225d82d673e P 7a4510b7224f42049881f3d65bd68685: Deleting consensus metadata
I20260925 10:57:43.740962 13964 catalog_manager.cc:4841] TS 7a4510b7224f42049881f3d65bd68685 (127.31.62.130:32815): tablet d51fb76e7f0f4e37b2fcb225d82d673e (table test-table [id=9d079940df4a4c4b80d8c22c9b2cea52]) successfully deleted
I20260925 10:57:43.829473 14473 tablet_replica.cc:330] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778: stopping tablet replica
I20260925 10:57:43.829975 14473 raft_consensus.cc:2229] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:57:43.830803 14473 raft_consensus.cc:2258] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:57:43.831828 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 14017
I20260925 10:57:43.850623 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 14170
W20260925 10:57:43.851495 13965 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.62.129:34229 (error 108)
I20260925 10:57:43.853240 14442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.852753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34229 (local address 127.0.0.1:41872)
0925 10:57:43.852943 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:43.853063 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:34229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 10:57:43.853488 13965 catalog_manager.cc:4584] TS 5423854314a740c5a7f9d497df7e9778 (127.31.62.129:34229): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d51fb76e7f0f4e37b2fcb225d82d673e: Network error: Client connection negotiation failed: client connection to 127.31.62.129:34229: connect: Connection refused (error 111)
I20260925 10:57:43.864609 14443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.852279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45728 (local address 127.31.62.130:32815)
0925 10:57:43.852452 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:43.852458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:43.852482 (+ 24us) server_negotiation.cc:407] Connection header received
0925 10:57:43.852714 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:43.852721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:43.852806 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:43.852950 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:43.854563 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.855292 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.856486 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.856877 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.856986 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.859341 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:43.859355 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:43.859370 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:43.859403 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:43.862076 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:43.862797 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:43.862805 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:43.862807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:43.862946 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:43.863287 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:43.863290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:43.863292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:43.863700 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:43.863902 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:43.864182 (+ 280us) server_negotiation.cc:299] Negotiation successful
0925 10:57:43.864346 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 10:57:43.865374 14294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:43.906970 14442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.906520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34229 (local address 127.0.0.1:41882)
0925 10:57:43.906732 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:43.906832 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:34229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260925 10:57:43.907934 14442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.907598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34229 (local address 127.0.0.1:41894)
0925 10:57:43.907739 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:43.907819 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:34229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 10:57:43.965471 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 14170
I20260925 10:57:43.968466 14442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.967884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34229 (local address 127.0.0.1:41910)
0925 10:57:43.968151 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:43.968263 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:34229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260925 10:57:43.969524 14442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.969143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34229 (local address 127.0.0.1:41920)
0925 10:57:43.969293 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:43.969376 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:34229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 10:57:43.980000 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 14302
I20260925 10:57:43.999639 14444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:43.980790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40404 (local address 127.31.62.131:42007)
0925 10:57:43.980994 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:43.981000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:43.981020 (+ 20us) server_negotiation.cc:407] Connection header received
0925 10:57:43.981208 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:43.981213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:43.981287 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:43.981373 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:43.982753 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.983483 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.984827 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:43.985255 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:43.985369 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:43.994132 (+ 8763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:43.994150 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:43.994166 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:43.994208 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:43.996793 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:43.997564 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:43.997573 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:43.997576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:43.997728 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:43.998096 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:43.998100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:43.998102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:43.998485 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:43.998780 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:43.999077 (+ 297us) server_negotiation.cc:299] Negotiation successful
0925 10:57:43.999303 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 10:57:44.000276 14426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:44.050233 14442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:44.049719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34229 (local address 127.0.0.1:41924)
0925 10:57:44.049951 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:44.050056 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:34229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260925 10:57:44.051258 14442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:44.050894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34229 (local address 127.0.0.1:41930)
0925 10:57:44.051074 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:44.051147 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:34229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260925 10:57:44.088671 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 14302
I20260925 10:57:44.101703 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 13946
I20260925 10:57:44.102764 14010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:57:44.216275 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 13946
I20260925 10:57:44.234618 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41871
--webserver_interface=127.31.62.190
--webserver_port=44107
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:41871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:57:44.559948 14479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:44.560259 14479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:44.560349 14479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:44.567831 14479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:57:44.567963 14479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:44.568010 14479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:44.568065 14479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:57:44.568101 14479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:57:44.578351 14479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:41871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=44107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:44.579627 14479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:44.580945 14479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:44.592500 14484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:44.593258 14487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:44.593331 14485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:44.593655 14479 server_base.cc:1034] running on GCE node
I20260925 10:57:44.594445 14479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:44.595978 14479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:44.597198 14479 hybrid_clock.cc:648] HybridClock initialized: now 1790333864597146 us; error 39 us; skew 500 ppm
I20260925 10:57:44.597596 14479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:44 dist-test-slave-nx7s krb5kdc[13395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333864, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:44.600663 14479 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:57:44.602231 14479 webserver.cc:460] Webserver started at http://127.31.62.190:44107/ using document root <none> and password file <none>
I20260925 10:57:44.602921 14479 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:44.603039 14479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:44.608821 14479 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 10:57:44.611754 14494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:44.613281 14479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 10:57:44.613481 14479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "0e68e89e7e734035b161024a74256219"
format_stamp: "Formatted at 2026-09-25 10:57:38 on dist-test-slave-nx7s"
server_key: "9fd4bda4585a30c38d63d86c082ae080"
server_key_iv: "ced6b6c69ba096503111472cb2974fb6"
server_key_version: "encryptionkey@0"
I20260925 10:57:44.614675 14479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:44.639627 14479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:44.644028 14479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:44.644279 14479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:44.661836 14479 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:41871
I20260925 10:57:44.661841 14547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:41871 every 8 connection(s)
I20260925 10:57:44.663388 14479 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:57:44.666848 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 14479
I20260925 10:57:44.667845 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:34229
--local_ip_for_outbound_sockets=127.31.62.129
--tserver_master_addrs=127.31.62.190:41871
--webserver_port=44759
--webserver_interface=127.31.62.129
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:57:44.671919 14548 sys_catalog.cc:263] Verifying existing consensus state
I20260925 10:57:44.676559 14548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: Bootstrap starting.
I20260925 10:57:44.696967 14548 log.cc:824] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:44.712822 14548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 10:57:44.713385 14548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219: Bootstrap complete.
I20260925 10:57:44.722652 14548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } }
I20260925 10:57:44.723531 14548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e68e89e7e734035b161024a74256219, State: Initialized, Role: FOLLOWER
I20260925 10:57:44.724130 14548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } }
I20260925 10:57:44.724349 14548 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:57:44.724457 14548 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:57:44.724627 14548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 2 FOLLOWER]: Advancing to term 3
I20260925 10:57:44.728494 14548 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } }
I20260925 10:57:44.729036 14548 leader_election.cc:302] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e68e89e7e734035b161024a74256219; no voters:
I20260925 10:57:44.729646 14548 leader_election.cc:288] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 10:57:44.729789 14552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 10:57:44.731809 14548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:57:44.733139 14552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [term 3 LEADER]: Becoming Leader. State: Replica: 0e68e89e7e734035b161024a74256219, State: Running, Role: LEADER
I20260925 10:57:44.733722 14552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } }
I20260925 10:57:44.736367 14554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e68e89e7e734035b161024a74256219. Latest consensus state: current_term: 3 leader_uuid: "0e68e89e7e734035b161024a74256219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } } }
I20260925 10:57:44.736752 14554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [sys.catalog]: This master's current role is: LEADER
I20260925 10:57:44.737325 14553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0e68e89e7e734035b161024a74256219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e68e89e7e734035b161024a74256219" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41871 } } }
I20260925 10:57:44.737596 14553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e68e89e7e734035b161024a74256219 [sys.catalog]: This master's current role is: LEADER
I20260925 10:57:44.740965 14558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:57:44.749598 14558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:57:44.751207 14558 catalog_manager.cc:1259] Loaded cluster ID: 2be48f33445145cea72e0353d86b563c
I20260925 10:57:44.751391 14558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260925 10:57:44.753441 14558 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4b817319ad google::LogMessage::Fail() at ??:0
@ 0x7f4b8173488a google::LogMessage::SendToLog() at ??:0
@ 0x7f4b817314de google::LogMessage::Flush() at ??:0
@ 0x7f4b81732a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4b97168512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f4b97165976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4b971ed610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f4b971ed439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4b8437d345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4b8438091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f4b84380799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4b8435beaf kudu::Thread::SuperviseThread() at ??:0
I20260925 10:57:44.815176 14575 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f4b876356db start_thread at ??:0
@ 0x7f4b805cb61f clone at ??:0
W20260925 10:57:45.054302 14550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:45.054598 14550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:45.054718 14550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:45.062173 14550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:45.062282 14550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:45.062440 14550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:57:45.072391 14550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:34229
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=44759
--tserver_master_addrs=127.31.62.190:41871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:45.073689 14550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:45.075009 14550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:45.089362 14593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:45.089388 14590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:45.089322 14591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:45.090265 14550 server_base.cc:1034] running on GCE node
I20260925 10:57:45.090900 14550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:45.091884 14550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:45.093114 14550 hybrid_clock.cc:648] HybridClock initialized: now 1790333865093056 us; error 52 us; skew 500 ppm
I20260925 10:57:45.093438 14550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:45 dist-test-slave-nx7s krb5kdc[13395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333865, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:45.096441 14550 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:57:45.097862 14550 webserver.cc:460] Webserver started at http://127.31.62.129:44759/ using document root <none> and password file <none>
I20260925 10:57:45.098517 14550 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:45.098616 14550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:45.104058 14550 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 10:57:45.107172 14600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:45.108675 14550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:57:45.108956 14550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "5423854314a740c5a7f9d497df7e9778"
format_stamp: "Formatted at 2026-09-25 10:57:39 on dist-test-slave-nx7s"
server_key: "a2d1be3111f43a29333c2698ac8d6ee5"
server_key_iv: "7056ee984f8c4dcf4843ef67fee40a7e"
server_key_version: "encryptionkey@0"
I20260925 10:57:45.109951 14550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:45.124768 14550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:45.129324 14550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:45.129583 14550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:45.130574 14550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:57:45.132469 14607 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260925 10:57:45.138299 14550 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 10:57:45.138438 14550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260925 10:57:45.138567 14550 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 10:57:45.141825 14550 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 10:57:45.141912 14550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260925 10:57:45.142254 14607 tablet_bootstrap.cc:492] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778: Bootstrap starting.
I20260925 10:57:45.153266 14607 log.cc:824] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778: Log is configured to *not* fsync() on all Append() calls
I20260925 10:57:45.168390 14607 tablet_bootstrap.cc:492] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 10:57:45.169221 14607 tablet_bootstrap.cc:492] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778: Bootstrap complete.
I20260925 10:57:45.169708 14607 ts_tablet_manager.cc:1397] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778: Time spent bootstrapping tablet: real 0.028s user 0.021s sys 0.004s
I20260925 10:57:45.177945 14550 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:34229
I20260925 10:57:45.177994 14715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:34229 every 8 connection(s)
I20260925 10:57:45.179368 14550 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:57:45.179327 14607 raft_consensus.cc:348] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:57:45.180799 14607 raft_consensus.cc:729] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5423854314a740c5a7f9d497df7e9778, State: Initialized, Role: FOLLOWER
I20260925 10:57:45.181641 14607 consensus_queue.cc:260] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:57:45.183354 14607 ts_tablet_manager.cc:1428] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260925 10:57:45.183665 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 14550
I20260925 10:57:45.184257 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:32815
--local_ip_for_outbound_sockets=127.31.62.130
--tserver_master_addrs=127.31.62.190:41871
--webserver_port=43865
--webserver_interface=127.31.62.130
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:57:45.185066 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:45.182572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:59337)
0925 10:57:45.183679 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:45.184064 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":391,"threads_started":1}
W20260925 10:57:45.185745 14716 heartbeater.cc:644] Failed to heartbeat to 127.31.62.190:41871 (0 consecutive failures): Network error: Failed to ping master at 127.31.62.190:41871: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
I20260925 10:57:45.186645 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:45.186293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:49201)
0925 10:57:45.186467 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:45.186538 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260925 10:57:45.187608 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:45.187220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:33321)
0925 10:57:45.187381 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:45.187459 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260925 10:57:45.187851 14716 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 10:57:45.508880 14720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:45.509163 14720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:45.509298 14720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:45.516390 14720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:45.516508 14720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:45.516672 14720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:57:45.526551 14720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:32815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=43865
--tserver_master_addrs=127.31.62.190:41871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:45.527902 14720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:45.529256 14720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:45.542097 14725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:45.542279 14726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:45.542089 14728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:45.542793 14720 server_base.cc:1034] running on GCE node
I20260925 10:57:45.543421 14720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:45.544355 14720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:45.545567 14720 hybrid_clock.cc:648] HybridClock initialized: now 1790333865545497 us; error 62 us; skew 500 ppm
I20260925 10:57:45.545881 14720 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:45 dist-test-slave-nx7s krb5kdc[13395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333865, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:45.548945 14720 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:57:45.550371 14720 webserver.cc:460] Webserver started at http://127.31.62.130:43865/ using document root <none> and password file <none>
I20260925 10:57:45.551050 14720 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:45.551167 14720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:45.556536 14720 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260925 10:57:45.559330 14735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:45.560771 14720 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 10:57:45.560951 14720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "7a4510b7224f42049881f3d65bd68685"
format_stamp: "Formatted at 2026-09-25 10:57:40 on dist-test-slave-nx7s"
server_key: "71214e07852e7309d7bd50b9182185fd"
server_key_iv: "ca54fa1bceaa23f1a7b9d8992ea9f039"
server_key_version: "encryptionkey@0"
I20260925 10:57:45.561954 14720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:45.585186 14720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:45.589236 14720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:45.589515 14720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:45.590529 14720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:57:45.591871 14720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:57:45.591956 14720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:45.592073 14720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:57:45.592135 14720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:45.622546 14720 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:32815
I20260925 10:57:45.622630 14848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:32815 every 8 connection(s)
I20260925 10:57:45.624044 14720 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:57:45.625782 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 14720
I20260925 10:57:45.626495 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:42007
--local_ip_for_outbound_sockets=127.31.62.131
--tserver_master_addrs=127.31.62.190:41871
--webserver_port=41849
--webserver_interface=127.31.62.131
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:57:45.629388 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:45.627303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:49411)
0925 10:57:45.628146 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:45.628425 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":315,"threads_started":1}
W20260925 10:57:45.630025 14849 heartbeater.cc:644] Failed to heartbeat to 127.31.62.190:41871 (0 consecutive failures): Network error: Failed to ping master at 127.31.62.190:41871: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
I20260925 10:57:45.630867 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:45.630553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:50173)
0925 10:57:45.630694 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:45.630764 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260925 10:57:45.631671 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:45.631361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:43091)
0925 10:57:45.631493 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:45.631566 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260925 10:57:45.631884 14849 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 10:57:45.946996 14852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:57:45.947292 14852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:57:45.947387 14852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:57:45.954715 14852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:57:45.954840 14852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:57:45.954970 14852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:57:45.964727 14852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:42007
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=41849
--tserver_master_addrs=127.31.62.190:41871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:57:45.966091 14852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:57:45.967425 14852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:57:45.980163 14858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:45.980134 14857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:57:45.980196 14860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:57:45.980914 14852 server_base.cc:1034] running on GCE node
I20260925 10:57:45.981559 14852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:57:45.982527 14852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:57:45.983757 14852 hybrid_clock.cc:648] HybridClock initialized: now 1790333865983696 us; error 48 us; skew 500 ppm
I20260925 10:57:45.984130 14852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:57:45 dist-test-slave-nx7s krb5kdc[13395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333865, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:57:45.987335 14852 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:57:45.988899 14852 webserver.cc:460] Webserver started at http://127.31.62.131:41849/ using document root <none> and password file <none>
I20260925 10:57:45.989601 14852 fs_manager.cc:359] Metadata directory not provided
I20260925 10:57:45.989719 14852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:57:45.995086 14852 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260925 10:57:45.997923 14867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:45.999394 14852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260925 10:57:45.999554 14852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "38921275dbb34d378e099ae3b33fe899"
format_stamp: "Formatted at 2026-09-25 10:57:40 on dist-test-slave-nx7s"
server_key: "c05ceadf40192d675f121e371e0ab0b1"
server_key_iv: "77c5ed51922dba831d0fe306a154b304"
server_key_version: "encryptionkey@0"
I20260925 10:57:46.000590 14852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:57:46.032999 14852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:57:46.037356 14852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:57:46.037619 14852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:57:46.038631 14852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:57:46.039973 14852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:57:46.040076 14852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:46.040191 14852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:57:46.040262 14852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:57:46.070569 14852 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:42007
I20260925 10:57:46.070627 14980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:42007 every 8 connection(s)
I20260925 10:57:46.072000 14852 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:57:46.076686 14983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:46.074898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:54847)
0925 10:57:46.075621 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:46.075840 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":292,"threads_started":1}
W20260925 10:57:46.077418 14981 heartbeater.cc:644] Failed to heartbeat to 127.31.62.190:41871 (0 consecutive failures): Network error: Failed to ping master at 127.31.62.190:41871: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
I20260925 10:57:46.078178 14983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:46.077876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:37551)
0925 10:57:46.078017 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:46.078078 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260925 10:57:46.078812 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 14852
I20260925 10:57:46.079027 14983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:46.078674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:54679)
0925 10:57:46.078811 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:46.078896 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260925 10:57:46.079289 14981 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 10:57:46.189227 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:46.188430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:56497)
0925 10:57:46.188914 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:46.189014 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":218,"threads_started":1}
I20260925 10:57:46.381027 14986 raft_consensus.cc:482] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:57:46.381346 14986 raft_consensus.cc:504] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:57:46.382985 14986 leader_election.cc:288] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a4510b7224f42049881f3d65bd68685 (127.31.62.130:32815), 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007)
Sep 25 10:57:46 dist-test-slave-nx7s krb5kdc[13395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333865, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
Sep 25 10:57:46 dist-test-slave-nx7s krb5kdc[13395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333865, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:57:46.404767 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:46.383449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:42007 (local address 127.31.62.129:33577)
0925 10:57:46.385960 (+ 2511us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:46.385979 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:46.386164 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:46.388381 (+ 2217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:46.388391 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:46.389254 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:46.390008 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:46.390030 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:46.391660 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:46.391664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:46.393077 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:46.393097 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:46.394001 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:46.394007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:46.394270 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:46.395053 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:46.395076 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:46.397822 (+ 2746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:46.401217 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:46.401227 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:46.401244 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:46.401776 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:46.402181 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:46.402186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:46.402190 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:46.402391 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:46.403137 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:46.403145 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:46.403488 (+ 343us) client_negotiation.cc:769] Sending connection context
0925 10:57:46.403736 (+ 248us) client_negotiation.cc:240] Negotiation successful
0925 10:57:46.404298 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2392,"mutex_wait_us":154,"spinlock_wait_cycles":2944,"thread_start_us":2192,"threads_started":1}
I20260925 10:57:46.404815 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:46.383715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:33577 (local address 127.31.62.131:42007)
0925 10:57:46.385560 (+ 1845us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:46.385574 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:46.386095 (+ 521us) server_negotiation.cc:407] Connection header received
0925 10:57:46.387661 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:46.387669 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:46.388000 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:46.388304 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:46.390199 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:46.391482 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:46.393211 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:46.393812 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:46.393957 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:46.398126 (+ 4169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:46.398135 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:46.398152 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:46.398193 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:46.401065 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:46.401908 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:46.401914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:46.401919 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:46.402074 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:46.402499 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:46.402502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:46.402504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:46.403007 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:46.403177 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:46.404049 (+ 872us) server_negotiation.cc:299] Negotiation successful
0925 10:57:46.404433 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1687,"thread_start_us":1538,"threads_started":1}
I20260925 10:57:46.405128 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:46.383483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:43801 (local address 127.31.62.130:32815)
0925 10:57:46.385035 (+ 1552us) server_negotiation.cc:206] Beginning negotiation
0925 10:57:46.385054 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 10:57:46.385506 (+ 452us) server_negotiation.cc:407] Connection header received
0925 10:57:46.387491 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:57:46.387500 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:57:46.387916 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:57:46.388185 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:57:46.390202 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:46.391482 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:46.393213 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:57:46.393842 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:57:46.394008 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:46.398319 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:57:46.398326 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:57:46.398339 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:57:46.398377 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:57:46.401183 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:46.401905 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:46.401912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:46.401918 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:46.402081 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:57:46.402513 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:57:46.402517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:57:46.402519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:57:46.403009 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:57:46.403181 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 10:57:46.404047 (+ 866us) server_negotiation.cc:299] Negotiation successful
0925 10:57:46.404390 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1305,"thread_start_us":985,"threads_started":1}
I20260925 10:57:46.404841 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:46.383418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:32815 (local address 127.31.62.129:43801)
0925 10:57:46.384955 (+ 1537us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:46.385000 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 10:57:46.385685 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:57:46.388380 (+ 2695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:57:46.388390 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:57:46.389254 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:57:46.390015 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:46.390030 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:46.391666 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:46.391670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:46.393079 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:57:46.393091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:57:46.394047 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:57:46.394054 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:57:46.394266 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:57:46.395064 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:57:46.395087 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:57:46.398052 (+ 2965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:57:46.401316 (+ 3264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:46.401321 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:46.401325 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:46.401765 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:46.402189 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:57:46.402194 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:57:46.402198 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:57:46.402400 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:57:46.403165 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:57:46.403169 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:57:46.403490 (+ 321us) client_negotiation.cc:769] Sending connection context
0925 10:57:46.403737 (+ 247us) client_negotiation.cc:240] Negotiation successful
0925 10:57:46.404453 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1302,"mutex_wait_us":473,"spinlock_wait_cycles":32000}
W20260925 10:57:46.408983 14603 leader_election.cc:341] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007): Not found: Tablet not found: d51fb76e7f0f4e37b2fcb225d82d673e
W20260925 10:57:46.409906 14602 leader_election.cc:341] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a4510b7224f42049881f3d65bd68685 (127.31.62.130:32815): Not found: Tablet not found: d51fb76e7f0f4e37b2fcb225d82d673e
I20260925 10:57:46.410033 14602 leader_election.cc:302] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5423854314a740c5a7f9d497df7e9778; no voters: 38921275dbb34d378e099ae3b33fe899, 7a4510b7224f42049881f3d65bd68685
I20260925 10:57:46.410425 14986 raft_consensus.cc:2726] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 10:57:46.633414 14990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:46.632503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:36231)
0925 10:57:46.633017 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:46.633175 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":234,"threads_started":1}
I20260925 10:57:47.080513 14991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:47.079791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:36813)
0925 10:57:47.080200 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:47.080306 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":195,"threads_started":1}
I20260925 10:57:47.190948 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:47.190156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:47371)
0925 10:57:47.190692 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:47.190777 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":284,"threads_started":1}
I20260925 10:57:47.635106 14993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:47.634371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:38691)
0925 10:57:47.634839 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:47.634926 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":227,"threads_started":1}
I20260925 10:57:48.082300 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:48.081499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:43365)
0925 10:57:48.082003 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:48.082095 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":286,"threads_started":1}
I20260925 10:57:48.192734 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:48.191935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:36343)
0925 10:57:48.192431 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:48.192554 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":252,"threads_started":1}
I20260925 10:57:48.462669 14996 raft_consensus.cc:482] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:57:48.462870 14996 raft_consensus.cc:504] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:57:48.463551 14996 leader_election.cc:288] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a4510b7224f42049881f3d65bd68685 (127.31.62.130:32815), 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007)
W20260925 10:57:48.464635 14602 leader_election.cc:341] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a4510b7224f42049881f3d65bd68685 (127.31.62.130:32815): Not found: Tablet not found: d51fb76e7f0f4e37b2fcb225d82d673e
W20260925 10:57:48.464866 14603 leader_election.cc:341] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007): Not found: Tablet not found: d51fb76e7f0f4e37b2fcb225d82d673e
I20260925 10:57:48.465052 14603 leader_election.cc:302] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5423854314a740c5a7f9d497df7e9778; no voters: 38921275dbb34d378e099ae3b33fe899, 7a4510b7224f42049881f3d65bd68685
I20260925 10:57:48.465365 14996 raft_consensus.cc:2726] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 10:57:48.636682 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:48.635985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:43123)
0925 10:57:48.636424 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:48.636504 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":230,"threads_started":1}
I20260925 10:57:49.083834 14998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:49.083145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:35103)
0925 10:57:49.083590 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:49.083676 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":214,"threads_started":1}
I20260925 10:57:49.194617 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:49.193757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:41495)
0925 10:57:49.194270 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:49.194380 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":248,"threads_started":1}
I20260925 10:57:49.638314 15000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:49.637584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:37105)
0925 10:57:49.638044 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:49.638128 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":218,"threads_started":1}
I20260925 10:57:50.085543 15001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:50.084730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:50669)
0925 10:57:50.085267 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:50.085370 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":288,"threads_started":1}
I20260925 10:57:50.196467 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:50.195718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:40399)
0925 10:57:50.196199 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:50.196283 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":211,"threads_started":1}
I20260925 10:57:50.640077 15003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:50.639322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:57183)
0925 10:57:50.639795 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:50.639885 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":254,"threads_started":1}
I20260925 10:57:51.087276 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:51.086548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:42247)
0925 10:57:51.087029 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:51.087122 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":246,"threads_started":1}
I20260925 10:57:51.198269 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:51.197442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:51099)
0925 10:57:51.197947 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:51.198067 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":257,"threads_started":1}
I20260925 10:57:51.531678 15006 raft_consensus.cc:482] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:57:51.531879 15006 raft_consensus.cc:504] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:57:51.532552 15006 leader_election.cc:288] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a4510b7224f42049881f3d65bd68685 (127.31.62.130:32815), 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007)
W20260925 10:57:51.533499 14602 leader_election.cc:341] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a4510b7224f42049881f3d65bd68685 (127.31.62.130:32815): Not found: Tablet not found: d51fb76e7f0f4e37b2fcb225d82d673e
W20260925 10:57:51.533658 14603 leader_election.cc:341] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007): Not found: Tablet not found: d51fb76e7f0f4e37b2fcb225d82d673e
I20260925 10:57:51.533797 14603 leader_election.cc:302] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5423854314a740c5a7f9d497df7e9778; no voters: 38921275dbb34d378e099ae3b33fe899, 7a4510b7224f42049881f3d65bd68685
I20260925 10:57:51.534024 15006 raft_consensus.cc:2726] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 10:57:51.641708 15007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:51.640982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:39789)
0925 10:57:51.641449 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:51.641528 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":253,"threads_started":1}
I20260925 10:57:52.089051 15008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:52.088207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:57583)
0925 10:57:52.088732 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:52.088824 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":287,"threads_started":1}
I20260925 10:57:52.200044 15009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:52.199296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:33979)
0925 10:57:52.199762 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:52.199851 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":226,"threads_started":1}
I20260925 10:57:52.643348 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:52.642565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:54461)
0925 10:57:52.643079 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:52.643168 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":238,"threads_started":1}
I20260925 10:57:53.090750 15011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:53.089937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:39809)
0925 10:57:53.090446 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:53.090524 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":215,"threads_started":1}
I20260925 10:57:53.202001 15012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:53.201075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:36663)
0925 10:57:53.201696 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:53.201800 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":284,"threads_started":1}
I20260925 10:57:53.645133 15013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:53.644290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:46759)
0925 10:57:53.644792 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:53.644930 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":270,"threads_started":1}
I20260925 10:57:54.092486 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:54.091822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:32807)
0925 10:57:54.092238 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:54.092321 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":211,"threads_started":1}
I20260925 10:57:54.203733 15015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:54.202972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:52077)
0925 10:57:54.203445 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:54.203531 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":236,"threads_started":1}
I20260925 10:57:54.646818 15016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:54.646095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:50465)
0925 10:57:54.646567 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:54.646663 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":220,"threads_started":1}
I20260925 10:57:55.094288 15017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:55.093448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:48471)
0925 10:57:55.093988 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:55.094121 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":263,"threads_started":1}
I20260925 10:57:55.205588 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:55.204749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:36159)
0925 10:57:55.205314 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:55.205420 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":326,"threads_started":1}
I20260925 10:57:55.461494 15020 raft_consensus.cc:482] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:57:55.461692 15020 raft_consensus.cc:504] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:57:55.462390 15020 leader_election.cc:288] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a4510b7224f42049881f3d65bd68685 (127.31.62.130:32815), 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007)
W20260925 10:57:55.463352 14602 leader_election.cc:341] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a4510b7224f42049881f3d65bd68685 (127.31.62.130:32815): Not found: Tablet not found: d51fb76e7f0f4e37b2fcb225d82d673e
W20260925 10:57:55.463522 14603 leader_election.cc:341] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007): Not found: Tablet not found: d51fb76e7f0f4e37b2fcb225d82d673e
I20260925 10:57:55.463670 14603 leader_election.cc:302] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5423854314a740c5a7f9d497df7e9778; no voters: 38921275dbb34d378e099ae3b33fe899, 7a4510b7224f42049881f3d65bd68685
I20260925 10:57:55.463889 15020 raft_consensus.cc:2726] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 10:57:55.648566 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:55.647862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:50601)
0925 10:57:55.648293 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:55.648381 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":219,"threads_started":1}
I20260925 10:57:56.096153 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:56.095393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:59209)
0925 10:57:56.095879 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:56.095975 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":256,"threads_started":1}
I20260925 10:57:56.207386 15023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:56.206574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:53253)
0925 10:57:56.207105 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:56.207197 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":261,"threads_started":1}
I20260925 10:57:56.650273 15024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:56.649497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:39977)
0925 10:57:56.650019 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:56.650116 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":250,"threads_started":1}
I20260925 10:57:57.098169 15025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:57.097341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:57201)
0925 10:57:57.097856 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:57.097974 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":258,"threads_started":1}
I20260925 10:57:57.209266 15026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:57.208450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:46951)
0925 10:57:57.208960 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:57.209060 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":242,"threads_started":1}
I20260925 10:57:57.652074 15027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:57.651293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:45525)
0925 10:57:57.651771 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:57.651861 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":249,"threads_started":1}
I20260925 10:57:58.100046 15028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:58.099282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:55955)
0925 10:57:58.099788 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:58.099876 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":270,"threads_started":1}
I20260925 10:57:58.211076 15029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:58.210306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:41619)
0925 10:57:58.210788 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:58.210896 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":260,"threads_started":1}
I20260925 10:57:58.653877 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:58.653164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:42877)
0925 10:57:58.653611 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:58.653703 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":218,"threads_started":1}
I20260925 10:57:59.101843 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:59.101012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:40477)
0925 10:57:59.101527 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:59.101631 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":245,"threads_started":1}
I20260925 10:57:59.213148 15032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:59.212172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:36607)
0925 10:57:59.212746 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:59.212902 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":289,"threads_started":1}
I20260925 10:57:59.655493 15033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:57:59.654744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:50273)
0925 10:57:59.655216 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0925 10:57:59.655303 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":245,"threads_started":1}
I20260925 10:58:00.103540 15034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:00.102764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:36201)
0925 10:58:00.103233 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:00.103347 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":244,"threads_started":1}
I20260925 10:58:00.215412 15035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:00.214474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.129:57933)
0925 10:58:00.215053 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:00.215176 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":252,"threads_started":1}
I20260925 10:58:00.657425 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:00.656636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.130:38431)
0925 10:58:00.657176 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:00.657269 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":275,"threads_started":1}
I20260925 10:58:01.083814 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 14550
I20260925 10:58:01.103566 15037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:01.084937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.31.62.129:34229)
0925 10:58:01.085479 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:01.085489 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:01.085520 (+ 31us) server_negotiation.cc:407] Connection header received
0925 10:58:01.085594 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:01.085604 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:01.085909 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:01.086088 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:01.088228 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:01.089347 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:01.090639 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:01.091195 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:01.091297 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:01.097748 (+ 6451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:01.097765 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:01.097781 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:01.097829 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:01.100652 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:01.101488 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:01.101496 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:01.101498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:01.101653 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:01.101995 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:01.101998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:01.102000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:01.102416 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:01.102659 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:01.103062 (+ 403us) server_negotiation.cc:299] Negotiation successful
0925 10:58:01.103286 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":235,"threads_started":1}
I20260925 10:58:01.105079 14711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:58:01.105347 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:01.104596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:41871 (local address 127.31.62.131:58175)
0925 10:58:01.105082 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:01.105173 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":258,"threads_started":1}
I20260925 10:58:01.197312 15040 raft_consensus.cc:482] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:58:01.197576 15040 raft_consensus.cc:504] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4510b7224f42049881f3d65bd68685" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 32815 } } peers { permanent_uuid: "38921275dbb34d378e099ae3b33fe899" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 42007 } } peers { permanent_uuid: "5423854314a740c5a7f9d497df7e9778" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34229 } }
I20260925 10:58:01.198328 15040 leader_election.cc:288] T d51fb76e7f0f4e37b2fcb225d82d673e P 5423854314a740c5a7f9d497df7e9778 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a4510b7224f42049881f3d65bd68685 (127.31.62.130:32815), 38921275dbb34d378e099ae3b33fe899 (127.31.62.131:42007)
I20260925 10:58:01.199203 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 14550
W20260925 10:58:01.214092 14736 connection.cc:535] server connection from 127.31.62.129:43801 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 10:58:01.214389 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 14720
I20260925 10:58:01.227716 15041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:01.215158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55520 (local address 127.31.62.130:32815)
0925 10:58:01.215635 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:01.215640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:01.215656 (+ 16us) server_negotiation.cc:407] Connection header received
0925 10:58:01.215711 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:01.215715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:01.215774 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:01.215875 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:01.217297 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:01.218084 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:01.219288 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:01.219707 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:01.219802 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:01.222457 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:01.222462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:01.222465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:01.222523 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:01.225075 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:01.225749 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:01.225754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:01.225757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:01.225979 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:01.226308 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:01.226311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:01.226313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:01.226613 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:01.226826 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:01.227119 (+ 293us) server_negotiation.cc:299] Negotiation successful
0925 10:58:01.227335 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":242,"threads_started":1}
I20260925 10:58:01.228394 14844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:58:01.312291 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 14720
I20260925 10:58:01.326248 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 14852
I20260925 10:58:01.339920 15043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:01.327182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51704 (local address 127.31.62.131:42007)
0925 10:58:01.327787 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:01.327792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:01.327809 (+ 17us) server_negotiation.cc:407] Connection header received
0925 10:58:01.327862 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:01.327866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:01.327988 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:01.328086 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:01.329507 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:01.330391 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:01.331686 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:01.332120 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:01.332221 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:01.334818 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:01.334823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:01.334826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:01.334861 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:01.337534 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:01.338297 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:01.338301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:01.338303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:01.338436 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:01.338731 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:01.338734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:01.338735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:01.338989 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:01.339206 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:01.339387 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 10:58:01.339555 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":313,"threads_started":1}
I20260925 10:58:01.340617 14976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:58:01.428764 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 14852
2026-09-25T10:58:01Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24122 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:58:01 dist-test-slave-nx7s krb5kdc[15047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:58:01 dist-test-slave-nx7s krb5kdc[15047](info): set up 2 sockets
Sep 25 10:58:01 dist-test-slave-nx7s krb5kdc[15047](info): commencing operation
krb5kdc: starting...
W20260925 10:58:02.457682 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:58:02 dist-test-slave-nx7s krb5kdc[15047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:58:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:58:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:58:02.689033 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:34111
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:34111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:58:03.022606 15060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:58:03.022933 15060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:58:03.023077 15060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:58:03.030447 15060 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 10:58:03.030567 15060 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 10:58:03.030625 15060 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 10:58:03.030694 15060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:58:03.030768 15060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:58:03.030864 15060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:58:03.030926 15060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:58:03.031009 15060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:58:03.041172 15060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:34111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:34111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:58:03.042500 15060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:58:03.043838 15060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:58:03.055536 15065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:03.055523 15068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:03.055545 15066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:58:03.056284 15060 server_base.cc:1034] running on GCE node
I20260925 10:58:03.057127 15060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:58:03.058547 15060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:58:03.059760 15060 hybrid_clock.cc:648] HybridClock initialized: now 1790333883059676 us; error 66 us; skew 500 ppm
I20260925 10:58:03.060077 15060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:58:03 dist-test-slave-nx7s krb5kdc[15047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333883, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:58:03.063161 15060 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:58:03.064694 15060 webserver.cc:460] Webserver started at http://127.31.62.190:37889/ using document root <none> and password file <none>
I20260925 10:58:03.065689 15060 fs_manager.cc:359] Metadata directory not provided
I20260925 10:58:03.065796 15060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:58:03.066071 15060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:58:03.069079 15060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5"
format_stamp: "Formatted at 2026-09-25 10:58:03 on dist-test-slave-nx7s"
server_key: "c23fd8b3a87778a20fbd5777e78f3d1f"
server_key_iv: "9822d4d372faf87b1bbbd4235598c1a6"
server_key_version: "encryptionkey@0"
I20260925 10:58:03.069835 15060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5"
format_stamp: "Formatted at 2026-09-25 10:58:03 on dist-test-slave-nx7s"
server_key: "c23fd8b3a87778a20fbd5777e78f3d1f"
server_key_iv: "9822d4d372faf87b1bbbd4235598c1a6"
server_key_version: "encryptionkey@0"
I20260925 10:58:03.074913 15060 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:58:03.078542 15075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:03.080018 15060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:58:03.080181 15060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5"
format_stamp: "Formatted at 2026-09-25 10:58:03 on dist-test-slave-nx7s"
server_key: "c23fd8b3a87778a20fbd5777e78f3d1f"
server_key_iv: "9822d4d372faf87b1bbbd4235598c1a6"
server_key_version: "encryptionkey@0"
I20260925 10:58:03.080385 15060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:58:03.108038 15060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:58:03.112280 15060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:58:03.112550 15060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:58:03.130041 15060 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:34111
I20260925 10:58:03.130082 15126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:34111 every 8 connection(s)
I20260925 10:58:03.131404 15060 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:58:03.135175 15127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:58:03.141403 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 15060
I20260925 10:58:03.141635 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:58:03.142457 31994 external_mini_cluster.cc:1428] Setting key e815f299825d528825977d5dcda51735
I20260925 10:58:03.147363 15127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Bootstrap starting.
I20260925 10:58:03.150611 15127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Neither blocks nor log segments found. Creating new log.
I20260925 10:58:03.151733 15127 log.cc:824] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Log is configured to *not* fsync() on all Append() calls
I20260925 10:58:03.154896 15127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: No bootstrap required, opened a new log
Sep 25 10:58:03 dist-test-slave-nx7s krb5kdc[15047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:58:03.162892 15127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } }
I20260925 10:58:03.163291 15127 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:58:03.163404 15127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa5d569e58449a3ad573f2cf3d3ffe5, State: Initialized, Role: FOLLOWER
I20260925 10:58:03.164019 15127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } }
I20260925 10:58:03.164264 15127 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:58:03.164448 15127 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:58:03.164662 15127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:58:03.166673 15130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:03.145018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38578 (local address 127.31.62.190:34111)
0925 10:58:03.145886 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:03.145899 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:03.145942 (+ 43us) server_negotiation.cc:407] Connection header received
0925 10:58:03.147741 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:03.147767 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:03.148190 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:03.148584 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:03.150092 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:03.151508 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:03.153278 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:03.153974 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:03.154147 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:03.158502 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:03.158511 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:03.158529 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:03.158569 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:03.161832 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:03.162590 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:03.162598 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:03.162606 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:03.162778 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:03.163180 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:03.163183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:03.163186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:03.163876 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:03.164166 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:03.164674 (+ 508us) server_negotiation.cc:299] Negotiation successful
0925 10:58:03.165205 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":417,"threads_started":1}
I20260925 10:58:03.168059 15127 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } }
I20260925 10:58:03.168617 15127 leader_election.cc:302] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1aa5d569e58449a3ad573f2cf3d3ffe5; no voters:
I20260925 10:58:03.169245 15127 leader_election.cc:288] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:58:03.169377 15132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:58:03.170580 15132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 1 LEADER]: Becoming Leader. State: Replica: 1aa5d569e58449a3ad573f2cf3d3ffe5, State: Running, Role: LEADER
I20260925 10:58:03.171192 15132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } }
I20260925 10:58:03.171934 15127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:58:03.173935 15134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aa5d569e58449a3ad573f2cf3d3ffe5. Latest consensus state: current_term: 1 leader_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } } }
I20260925 10:58:03.174376 15134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:58:03.173909 15133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } } }
I20260925 10:58:03.174872 15133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:58:03.176434 15137 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:58:03.182421 15137 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:58:03.191882 15137 catalog_manager.cc:1347] Generated new cluster ID: 76e51fd4b12043608d673ea1428510ee
I20260925 10:58:03.192020 15137 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:58:03.207755 15137 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:58:03.209734 15137 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:58:03.216944 15137 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Generated new TSK 0
I20260925 10:58:03.218091 15137 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:58:03.285072 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34111
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:58:03.622902 15154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:58:03.623193 15154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:58:03.623317 15154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:58:03.630522 15154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:58:03.630630 15154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:58:03.630779 15154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:58:03.640347 15154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:58:03.641623 15154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:58:03.642899 15154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:58:03.655506 15159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:03.655550 15162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:03.655550 15160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:58:03.656445 15154 server_base.cc:1034] running on GCE node
I20260925 10:58:03.657090 15154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:58:03.658016 15154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:58:03.659210 15154 hybrid_clock.cc:648] HybridClock initialized: now 1790333883659154 us; error 46 us; skew 500 ppm
I20260925 10:58:03.659503 15154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:58:03 dist-test-slave-nx7s krb5kdc[15047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333883, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:58:03.662482 15154 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:58:03.663831 15154 webserver.cc:460] Webserver started at http://127.31.62.129:46689/ using document root <none> and password file <none>
I20260925 10:58:03.664506 15154 fs_manager.cc:359] Metadata directory not provided
I20260925 10:58:03.664633 15154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:58:03.664963 15154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:58:03.667913 15154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "0b1719c04ff84ab59df62b3e866578ea"
format_stamp: "Formatted at 2026-09-25 10:58:03 on dist-test-slave-nx7s"
server_key: "bbf81a67656c5fa9c5534108de43e8cc"
server_key_iv: "3c1aa285889b4c2a189d39d3b3bba724"
server_key_version: "encryptionkey@0"
I20260925 10:58:03.668678 15154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "0b1719c04ff84ab59df62b3e866578ea"
format_stamp: "Formatted at 2026-09-25 10:58:03 on dist-test-slave-nx7s"
server_key: "bbf81a67656c5fa9c5534108de43e8cc"
server_key_iv: "3c1aa285889b4c2a189d39d3b3bba724"
server_key_version: "encryptionkey@0"
I20260925 10:58:03.674315 15154 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:58:03.677795 15169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:03.679196 15154 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:58:03.679350 15154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "0b1719c04ff84ab59df62b3e866578ea"
format_stamp: "Formatted at 2026-09-25 10:58:03 on dist-test-slave-nx7s"
server_key: "bbf81a67656c5fa9c5534108de43e8cc"
server_key_iv: "3c1aa285889b4c2a189d39d3b3bba724"
server_key_version: "encryptionkey@0"
I20260925 10:58:03.679561 15154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:58:03.696247 15154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:58:03.700436 15154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:58:03.700702 15154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:58:03.701674 15154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:58:03.702951 15154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:58:03.703042 15154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:03.703177 15154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:58:03.703234 15154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:03.733094 15154 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:44585
I20260925 10:58:03.733137 15281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:44585 every 8 connection(s)
I20260925 10:58:03.734351 15154 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:58:03.740456 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 15154
I20260925 10:58:03.740688 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:58:03.741485 31994 external_mini_cluster.cc:1428] Setting key 91d2304d4f467583ef796b22f469c2e6
Sep 25 10:58:03 dist-test-slave-nx7s krb5kdc[15047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333883, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:58:03.755908 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:03.736810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:37821 (local address 127.31.62.190:34111)
0925 10:58:03.737266 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:03.737272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:03.738295 (+ 1023us) server_negotiation.cc:407] Connection header received
0925 10:58:03.740103 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:03.740107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:03.740174 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:03.740272 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:03.742332 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:03.743344 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:03.745553 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:03.745958 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:03.746060 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:03.750179 (+ 4119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:03.750185 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:03.750189 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:03.750223 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:03.752749 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:03.753572 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:03.753579 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:03.753583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:03.753730 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:03.754143 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:03.754147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:03.754150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:03.754489 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:03.754634 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:03.755364 (+ 730us) server_negotiation.cc:299] Negotiation successful
0925 10:58:03.755582 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":220,"threads_started":1}
I20260925 10:58:03.756767 15285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:03.737429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:37821)
0925 10:58:03.738132 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:03.738176 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:03.739820 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:03.740557 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:03.740570 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:03.741343 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:58:03.742155 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:03.742177 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:03.743485 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:03.743491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:03.745410 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:03.745426 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:03.746179 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:03.746186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:03.746449 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:03.747219 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:58:03.747253 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:58:03.749903 (+ 2650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:58:03.752944 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:58:03.752951 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:58:03.752967 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:58:03.753447 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:58:03.753821 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:58:03.753824 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:58:03.753827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:58:03.754041 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:58:03.754612 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:58:03.754618 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:58:03.754953 (+ 335us) client_negotiation.cc:769] Sending connection context
0925 10:58:03.755236 (+ 283us) client_negotiation.cc:240] Negotiation successful
0925 10:58:03.755636 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":270,"threads_started":1}
I20260925 10:58:03.758410 15282 heartbeater.cc:347] Connected to a master server at 127.31.62.190:34111
I20260925 10:58:03.758708 15282 heartbeater.cc:464] Registering TS with master...
I20260925 10:58:03.759454 15282 heartbeater.cc:511] Master 127.31.62.190:34111 requested a full tablet report, sending...
I20260925 10:58:03.761685 15092 ts_manager.cc:194] Registered new tserver with Master: 0b1719c04ff84ab59df62b3e866578ea (127.31.62.129:44585)
I20260925 10:58:03.763159 15092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:37821
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:58:03.795760 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34111
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:58:04.118086 15288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:58:04.118394 15288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:58:04.118543 15288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:58:04.125962 15288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:58:04.126094 15288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:58:04.126250 15288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:58:04.140321 15288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:58:04.141857 15288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:58:04.143467 15288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:58:04.157955 15293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:04.157836 15296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:04.157896 15294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:58:04.158797 15288 server_base.cc:1034] running on GCE node
I20260925 10:58:04.159348 15288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:58:04.160254 15288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:58:04.161440 15288 hybrid_clock.cc:648] HybridClock initialized: now 1790333884161379 us; error 50 us; skew 500 ppm
I20260925 10:58:04.161758 15288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:58:04 dist-test-slave-nx7s krb5kdc[15047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333884, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:58:04.165102 15288 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:58:04.166539 15288 webserver.cc:460] Webserver started at http://127.31.62.130:45021/ using document root <none> and password file <none>
I20260925 10:58:04.167207 15288 fs_manager.cc:359] Metadata directory not provided
I20260925 10:58:04.167304 15288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:58:04.167551 15288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:58:04.170426 15288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "3334a8fea2ea46bbadf2f9c28cd5e268"
format_stamp: "Formatted at 2026-09-25 10:58:04 on dist-test-slave-nx7s"
server_key: "1ab346a62880dec4b7f75f55e816c0fd"
server_key_iv: "62e19caf289c28aae032315ea65244f1"
server_key_version: "encryptionkey@0"
I20260925 10:58:04.171144 15288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "3334a8fea2ea46bbadf2f9c28cd5e268"
format_stamp: "Formatted at 2026-09-25 10:58:04 on dist-test-slave-nx7s"
server_key: "1ab346a62880dec4b7f75f55e816c0fd"
server_key_iv: "62e19caf289c28aae032315ea65244f1"
server_key_version: "encryptionkey@0"
I20260925 10:58:04.176338 15288 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 10:58:04.180166 15303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:04.181576 15288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:58:04.181712 15288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "3334a8fea2ea46bbadf2f9c28cd5e268"
format_stamp: "Formatted at 2026-09-25 10:58:04 on dist-test-slave-nx7s"
server_key: "1ab346a62880dec4b7f75f55e816c0fd"
server_key_iv: "62e19caf289c28aae032315ea65244f1"
server_key_version: "encryptionkey@0"
I20260925 10:58:04.181878 15288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:58:04.203030 15288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:58:04.207504 15288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:58:04.207762 15288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:58:04.208721 15288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:58:04.210158 15288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:58:04.210258 15288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:04.210361 15288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:58:04.210403 15288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:04.240628 15288 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:35557
I20260925 10:58:04.240667 15417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:35557 every 8 connection(s)
I20260925 10:58:04.241932 15288 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:58:04.247260 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 15288
I20260925 10:58:04.247429 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:58:04.247879 31994 external_mini_cluster.cc:1428] Setting key 30996c8c02aaf4ee9ddd757fc23cead7
Sep 25 10:58:04 dist-test-slave-nx7s krb5kdc[15047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333884, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:58:04.263063 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.244533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:54375 (local address 127.31.62.190:34111)
0925 10:58:04.244764 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:04.244770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:04.246082 (+ 1312us) server_negotiation.cc:407] Connection header received
0925 10:58:04.248069 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:04.248076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:04.248173 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:04.248291 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:04.250095 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.251109 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.252706 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.253296 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.253419 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.257216 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:04.257223 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:04.257228 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:04.257274 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:04.259769 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:04.260715 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:04.260719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:04.260721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:04.260895 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:04.261348 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:04.261351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:04.261353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:04.261618 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:04.261761 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:04.262632 (+ 871us) server_negotiation.cc:299] Negotiation successful
0925 10:58:04.262816 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260925 10:58:04.264564 15420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.245151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:54375)
0925 10:58:04.245907 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:04.245953 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:04.247731 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:04.248530 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:04.248543 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:04.249199 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:58:04.249913 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:04.249942 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.251266 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.251270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:04.252564 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:04.252579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.253545 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.253551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:04.253770 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.254543 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:58:04.254572 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:58:04.256943 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:58:04.259931 (+ 2988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:58:04.259941 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:58:04.259961 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:58:04.260587 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:58:04.260992 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:58:04.260996 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:58:04.260999 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:58:04.261249 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:58:04.261764 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:58:04.261789 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:58:04.262245 (+ 456us) client_negotiation.cc:769] Sending connection context
0925 10:58:04.262543 (+ 298us) client_negotiation.cc:240] Negotiation successful
0925 10:58:04.263089 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":277,"threads_started":1}
I20260925 10:58:04.266079 15418 heartbeater.cc:347] Connected to a master server at 127.31.62.190:34111
I20260925 10:58:04.266412 15418 heartbeater.cc:464] Registering TS with master...
I20260925 10:58:04.267164 15418 heartbeater.cc:511] Master 127.31.62.190:34111 requested a full tablet report, sending...
I20260925 10:58:04.268680 15092 ts_manager.cc:194] Registered new tserver with Master: 3334a8fea2ea46bbadf2f9c28cd5e268 (127.31.62.130:35557)
I20260925 10:58:04.269511 15092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:54375
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:58:04.299665 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34111
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:58:04.619901 15423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:58:04.620208 15423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:58:04.620337 15423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:58:04.627707 15423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:58:04.627849 15423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:58:04.628010 15423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:58:04.637838 15423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:34111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:58:04.639145 15423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:58:04.640453 15423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:58:04.653529 15429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:04.653560 15428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:04.654104 15431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:58:04.654505 15423 server_base.cc:1034] running on GCE node
I20260925 10:58:04.655122 15423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:58:04.656157 15423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:58:04.657346 15423 hybrid_clock.cc:648] HybridClock initialized: now 1790333884657291 us; error 60 us; skew 500 ppm
I20260925 10:58:04.657658 15423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:58:04 dist-test-slave-nx7s krb5kdc[15047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333884, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:58:04.660987 15423 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:58:04.662531 15423 webserver.cc:460] Webserver started at http://127.31.62.131:43259/ using document root <none> and password file <none>
I20260925 10:58:04.663241 15423 fs_manager.cc:359] Metadata directory not provided
I20260925 10:58:04.663345 15423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:58:04.663632 15423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:58:04.666730 15423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "32389eab881f4599b46dba9d30079bc1"
format_stamp: "Formatted at 2026-09-25 10:58:04 on dist-test-slave-nx7s"
server_key: "dfa42167be4081a996b9041a7ce50ac1"
server_key_iv: "bebe7e8b8c7e5ff28f9714f3fcf26bb8"
server_key_version: "encryptionkey@0"
I20260925 10:58:04.667486 15423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "32389eab881f4599b46dba9d30079bc1"
format_stamp: "Formatted at 2026-09-25 10:58:04 on dist-test-slave-nx7s"
server_key: "dfa42167be4081a996b9041a7ce50ac1"
server_key_iv: "bebe7e8b8c7e5ff28f9714f3fcf26bb8"
server_key_version: "encryptionkey@0"
I20260925 10:58:04.673012 15423 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260925 10:58:04.676932 15438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:04.678421 15423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:58:04.678606 15423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "32389eab881f4599b46dba9d30079bc1"
format_stamp: "Formatted at 2026-09-25 10:58:04 on dist-test-slave-nx7s"
server_key: "dfa42167be4081a996b9041a7ce50ac1"
server_key_iv: "bebe7e8b8c7e5ff28f9714f3fcf26bb8"
server_key_version: "encryptionkey@0"
I20260925 10:58:04.678817 15423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:58:04.700088 15423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:58:04.704286 15423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:58:04.704557 15423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:58:04.705621 15423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:58:04.706956 15423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:58:04.707036 15423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:04.707204 15423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:58:04.707265 15423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:04.737967 15423 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:43497
I20260925 10:58:04.738010 15551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:43497 every 8 connection(s)
I20260925 10:58:04.739588 15423 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:58:04.741554 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 15423
I20260925 10:58:04.741745 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:58:04.742372 31994 external_mini_cluster.cc:1428] Setting key f58e0b4d946aab83bc932e3056cf20eb
Sep 25 10:58:04 dist-test-slave-nx7s krb5kdc[15047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333884, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:58:04.760229 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.742381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:44481 (local address 127.31.62.190:34111)
0925 10:58:04.742544 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:04.742549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:04.744207 (+ 1658us) server_negotiation.cc:407] Connection header received
0925 10:58:04.746474 (+ 2267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:04.746480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:04.746590 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:04.746674 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:04.748243 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.749031 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.750570 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.750952 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.751059 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.754496 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:04.754502 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:04.754505 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:04.754538 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:04.757018 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:04.757987 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:04.757992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:04.757995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:04.758157 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:04.758723 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:04.758726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:04.758728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:04.759000 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:04.759113 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:04.759818 (+ 705us) server_negotiation.cc:299] Negotiation successful
0925 10:58:04.759988 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 10:58:04.761355 15554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.743095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:44481)
0925 10:58:04.744038 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:04.744109 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:04.746208 (+ 2099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:04.746930 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:04.746939 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:04.747493 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:58:04.748074 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:04.748093 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.749175 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.749180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:04.750421 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:04.750434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.751178 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.751186 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:04.751405 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.752046 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:58:04.752085 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:58:04.754286 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:58:04.757148 (+ 2862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:58:04.757158 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:58:04.757174 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:58:04.757776 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:58:04.758300 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:58:04.758306 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:58:04.758310 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:58:04.758614 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:58:04.759108 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:58:04.759115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:58:04.759494 (+ 379us) client_negotiation.cc:769] Sending connection context
0925 10:58:04.759748 (+ 254us) client_negotiation.cc:240] Negotiation successful
0925 10:58:04.760165 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":410,"threads_started":1}
I20260925 10:58:04.762807 15552 heartbeater.cc:347] Connected to a master server at 127.31.62.190:34111
I20260925 10:58:04.763134 15552 heartbeater.cc:464] Registering TS with master...
I20260925 10:58:04.763826 15552 heartbeater.cc:511] Master 127.31.62.190:34111 requested a full tablet report, sending...
I20260925 10:58:04.765290 15092 ts_manager.cc:194] Registered new tserver with Master: 32389eab881f4599b46dba9d30079bc1 (127.31.62.131:43497)
I20260925 10:58:04.766100 15092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:44481
I20260925 10:58:04.766162 15282 heartbeater.cc:503] Master 127.31.62.190:34111 was elected leader, sending a full tablet report...
I20260925 10:58:04.770748 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:58:04.788106 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.775157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38588 (local address 127.31.62.190:34111)
0925 10:58:04.775390 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:04.775396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:04.775721 (+ 325us) server_negotiation.cc:407] Connection header received
0925 10:58:04.775893 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:04.775898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:04.775961 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:04.776040 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:04.777606 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.778335 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.779742 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.780144 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.780254 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.782636 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:04.782641 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:04.782645 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:04.782686 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:04.785419 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:04.786112 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:04.786117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:04.786119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:04.786251 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:04.786610 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:04.786613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:04.786615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:04.787006 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:04.787242 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:04.787575 (+ 333us) server_negotiation.cc:299] Negotiation successful
0925 10:58:04.787784 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 10:58:04.792214 15092 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:58:04.799468 15092 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:58:04.831151 15565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.812489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:44585 (local address 127.0.0.1:34902)
0925 10:58:04.823253 (+ 10764us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:04.823270 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:04.823403 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:04.824213 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:04.824218 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:04.824261 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:58:04.824779 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:04.824789 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.826603 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.826608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:04.828287 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:04.828300 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.830050 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.830056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:04.830261 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.830322 (+ 61us) client_negotiation.cc:769] Sending connection context
0925 10:58:04.830463 (+ 141us) client_negotiation.cc:240] Negotiation successful
0925 10:58:04.830664 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10640,"mutex_wait_us":38,"spinlock_wait_cycles":6144,"thread_start_us":8559,"threads_started":1}
I20260925 10:58:04.831472 15568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.812757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41922 (local address 127.31.62.131:43497)
0925 10:58:04.822797 (+ 10040us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:04.822805 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:04.822844 (+ 39us) server_negotiation.cc:407] Connection header received
0925 10:58:04.822956 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:04.822965 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:04.823274 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:04.824225 (+ 951us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:58:04.824521 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.826181 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.828825 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.830241 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.830378 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.830606 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:04.830759 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 10:58:04.830936 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9870,"thread_start_us":9701,"threads_started":1}
I20260925 10:58:04.831745 15566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.812098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:35557 (local address 127.0.0.1:50700)
0925 10:58:04.822887 (+ 10789us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:04.822906 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:04.823043 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:04.824213 (+ 1170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:04.824218 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:04.824252 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:58:04.824735 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:04.824751 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.826125 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.826131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:04.828638 (+ 2507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:04.828653 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.830910 (+ 2257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.830920 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:04.831102 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.831140 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 10:58:04.831230 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 10:58:04.831352 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10609,"mutex_wait_us":49,"spinlock_wait_cycles":1792,"thread_start_us":10548,"threads_started":1}
I20260925 10:58:04.831765 15569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.813083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34902 (local address 127.31.62.129:44585)
0925 10:58:04.820884 (+ 7801us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:04.820895 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:04.823415 (+ 2520us) server_negotiation.cc:407] Connection header received
0925 10:58:04.823522 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:04.823529 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:04.823748 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:04.823951 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:58:04.824965 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.826462 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.828442 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.829860 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.829982 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.830232 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:04.831291 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0925 10:58:04.831465 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7545,"thread_start_us":7380,"threads_started":1}
I20260925 10:58:04.831192 15567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.812075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:43497 (local address 127.0.0.1:41922)
0925 10:58:04.821486 (+ 9411us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:04.821546 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:04.821751 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:04.823526 (+ 1775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:04.823534 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:04.823571 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:58:04.824012 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:04.824025 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.826369 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.826375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:04.828638 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:04.828653 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.830435 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.830442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:04.830591 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.830622 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 10:58:04.830704 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 10:58:04.830819 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9112,"mutex_wait_us":51,"thread_start_us":10889,"threads_started":1}
I20260925 10:58:04.832484 15564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.812108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50700 (local address 127.31.62.130:35557)
0925 10:58:04.822391 (+ 10283us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:04.822401 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:04.823179 (+ 778us) server_negotiation.cc:407] Connection header received
0925 10:58:04.823263 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:04.823268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:04.823749 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:04.823951 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:58:04.825003 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.825995 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.828816 (+ 2821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.830456 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.830576 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.830751 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:04.832028 (+ 1277us) server_negotiation.cc:299] Negotiation successful
0925 10:58:04.832188 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10104,"thread_start_us":9972,"threads_started":1}
I20260925 10:58:04.835582 15486 tablet_service.cc:1467] Processing CreateTablet for tablet e23008b3710f43c2b92c24ba2d02365c (DEFAULT_TABLE table=test-table [id=006f6f47c9ee4f11b4ff37e96e086a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:58:04.835593 15351 tablet_service.cc:1467] Processing CreateTablet for tablet e23008b3710f43c2b92c24ba2d02365c (DEFAULT_TABLE table=test-table [id=006f6f47c9ee4f11b4ff37e96e086a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:58:04.835584 15217 tablet_service.cc:1467] Processing CreateTablet for tablet e23008b3710f43c2b92c24ba2d02365c (DEFAULT_TABLE table=test-table [id=006f6f47c9ee4f11b4ff37e96e086a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:58:04.836966 15486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e23008b3710f43c2b92c24ba2d02365c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:58:04.836992 15351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e23008b3710f43c2b92c24ba2d02365c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:58:04.836993 15217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e23008b3710f43c2b92c24ba2d02365c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:58:04.848662 15570 tablet_bootstrap.cc:492] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1: Bootstrap starting.
I20260925 10:58:04.851423 15571 tablet_bootstrap.cc:492] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea: Bootstrap starting.
I20260925 10:58:04.853013 15572 tablet_bootstrap.cc:492] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268: Bootstrap starting.
I20260925 10:58:04.854068 15570 tablet_bootstrap.cc:654] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1: Neither blocks nor log segments found. Creating new log.
I20260925 10:58:04.855247 15571 tablet_bootstrap.cc:654] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea: Neither blocks nor log segments found. Creating new log.
I20260925 10:58:04.855366 15570 log.cc:824] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1: Log is configured to *not* fsync() on all Append() calls
I20260925 10:58:04.855839 15572 tablet_bootstrap.cc:654] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268: Neither blocks nor log segments found. Creating new log.
I20260925 10:58:04.856402 15571 log.cc:824] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea: Log is configured to *not* fsync() on all Append() calls
I20260925 10:58:04.857122 15572 log.cc:824] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268: Log is configured to *not* fsync() on all Append() calls
I20260925 10:58:04.858651 15570 tablet_bootstrap.cc:492] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1: No bootstrap required, opened a new log
I20260925 10:58:04.858949 15570 ts_tablet_manager.cc:1397] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260925 10:58:04.859409 15571 tablet_bootstrap.cc:492] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea: No bootstrap required, opened a new log
I20260925 10:58:04.859663 15571 ts_tablet_manager.cc:1397] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260925 10:58:04.863013 15572 tablet_bootstrap.cc:492] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268: No bootstrap required, opened a new log
I20260925 10:58:04.863292 15572 ts_tablet_manager.cc:1397] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260925 10:58:04.870139 15570 raft_consensus.cc:348] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } }
I20260925 10:58:04.870110 15571 raft_consensus.cc:348] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } }
I20260925 10:58:04.870609 15571 raft_consensus.cc:374] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:58:04.870609 15570 raft_consensus.cc:374] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:58:04.870445 15572 raft_consensus.cc:348] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } }
I20260925 10:58:04.870708 15570 raft_consensus.cc:729] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32389eab881f4599b46dba9d30079bc1, State: Initialized, Role: FOLLOWER
I20260925 10:58:04.870738 15571 raft_consensus.cc:729] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1719c04ff84ab59df62b3e866578ea, State: Initialized, Role: FOLLOWER
I20260925 10:58:04.870870 15572 raft_consensus.cc:374] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:58:04.870965 15572 raft_consensus.cc:729] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3334a8fea2ea46bbadf2f9c28cd5e268, State: Initialized, Role: FOLLOWER
I20260925 10:58:04.871490 15572 consensus_queue.cc:260] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } }
I20260925 10:58:04.871445 15570 consensus_queue.cc:260] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } }
I20260925 10:58:04.871445 15571 consensus_queue.cc:260] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } }
I20260925 10:58:04.873766 15552 heartbeater.cc:503] Master 127.31.62.190:34111 was elected leader, sending a full tablet report...
I20260925 10:58:04.873890 15418 heartbeater.cc:503] Master 127.31.62.190:34111 was elected leader, sending a full tablet report...
I20260925 10:58:04.874265 15570 ts_tablet_manager.cc:1428] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1: Time spent starting tablet: real 0.015s user 0.013s sys 0.002s
I20260925 10:58:04.874662 15571 ts_tablet_manager.cc:1428] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea: Time spent starting tablet: real 0.015s user 0.009s sys 0.007s
I20260925 10:58:04.875049 15572 ts_tablet_manager.cc:1428] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20260925 10:58:04.891826 15577 raft_consensus.cc:482] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:58:04.892146 15577 raft_consensus.cc:504] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } }
I20260925 10:58:04.893989 15577 leader_election.cc:288] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3334a8fea2ea46bbadf2f9c28cd5e268 (127.31.62.130:35557), 0b1719c04ff84ab59df62b3e866578ea (127.31.62.129:44585)
I20260925 10:58:04.899259 15564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.894449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:60749 (local address 127.31.62.130:35557)
0925 10:58:04.894634 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:04.894640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:04.894657 (+ 17us) server_negotiation.cc:407] Connection header received
0925 10:58:04.894717 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:04.894720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:04.894774 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:04.894842 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:58:04.895220 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.895934 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.897625 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.898439 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.898586 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.898660 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:04.898899 (+ 239us) server_negotiation.cc:299] Negotiation successful
0925 10:58:04.899018 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 10:58:04.899266 15554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.894173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:35557 (local address 127.31.62.131:60749)
0925 10:58:04.894397 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:04.894418 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:04.894547 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:04.894892 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:04.894896 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:04.894925 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:58:04.895094 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:04.895103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.896067 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.896072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:04.897468 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:04.897481 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.898580 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.898586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:04.898749 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.898772 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 10:58:04.898852 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 10:58:04.898977 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260925 10:58:04.899981 15569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.894513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:44647 (local address 127.31.62.129:44585)
0925 10:58:04.894782 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:04.894788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:04.895178 (+ 390us) server_negotiation.cc:407] Connection header received
0925 10:58:04.895330 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:04.895334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:04.895390 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:04.895463 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:58:04.895978 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.896669 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.898291 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.899051 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.899201 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.899270 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:04.899601 (+ 331us) server_negotiation.cc:299] Negotiation successful
0925 10:58:04.899734 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260925 10:58:04.899976 15579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.894390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:44585 (local address 127.31.62.131:44647)
0925 10:58:04.895079 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:04.895093 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:04.895206 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:04.895483 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:04.895487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:04.895508 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:58:04.895836 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:04.895844 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.896809 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.896812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:04.898146 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:04.898158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.899244 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.899254 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:04.899427 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.899462 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 10:58:04.899549 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 10:58:04.899677 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":3,"thread_start_us":392,"threads_started":1}
I20260925 10:58:04.900403 15373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23008b3710f43c2b92c24ba2d02365c" candidate_uuid: "32389eab881f4599b46dba9d30079bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" is_pre_election: true
I20260925 10:58:04.900921 15373 raft_consensus.cc:2454] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32389eab881f4599b46dba9d30079bc1 in term 0.
I20260925 10:58:04.900976 15237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23008b3710f43c2b92c24ba2d02365c" candidate_uuid: "32389eab881f4599b46dba9d30079bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1719c04ff84ab59df62b3e866578ea" is_pre_election: true
I20260925 10:58:04.901484 15237 raft_consensus.cc:2454] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32389eab881f4599b46dba9d30079bc1 in term 0.
I20260925 10:58:04.901569 15442 leader_election.cc:302] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32389eab881f4599b46dba9d30079bc1, 3334a8fea2ea46bbadf2f9c28cd5e268; no voters:
I20260925 10:58:04.901965 15577 raft_consensus.cc:2781] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:58:04.902100 15577 raft_consensus.cc:482] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:58:04.902200 15577 raft_consensus.cc:3037] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:58:04.904351 15577 raft_consensus.cc:504] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } }
I20260925 10:58:04.905059 15577 leader_election.cc:288] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [CANDIDATE]: Term 1 election: Requested vote from peers 3334a8fea2ea46bbadf2f9c28cd5e268 (127.31.62.130:35557), 0b1719c04ff84ab59df62b3e866578ea (127.31.62.129:44585)
I20260925 10:58:04.905472 15373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23008b3710f43c2b92c24ba2d02365c" candidate_uuid: "32389eab881f4599b46dba9d30079bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268"
I20260925 10:58:04.905546 15237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23008b3710f43c2b92c24ba2d02365c" candidate_uuid: "32389eab881f4599b46dba9d30079bc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1719c04ff84ab59df62b3e866578ea"
I20260925 10:58:04.905678 15373 raft_consensus.cc:3037] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:58:04.905726 15237 raft_consensus.cc:3037] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:58:04.907838 15237 raft_consensus.cc:2454] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32389eab881f4599b46dba9d30079bc1 in term 1.
I20260925 10:58:04.907858 15373 raft_consensus.cc:2454] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32389eab881f4599b46dba9d30079bc1 in term 1.
I20260925 10:58:04.908267 15439 leader_election.cc:302] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1719c04ff84ab59df62b3e866578ea, 32389eab881f4599b46dba9d30079bc1; no voters:
I20260925 10:58:04.908594 15577 raft_consensus.cc:2781] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:58:04.909027 15577 raft_consensus.cc:686] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [term 1 LEADER]: Becoming Leader. State: Replica: 32389eab881f4599b46dba9d30079bc1, State: Running, Role: LEADER
I20260925 10:58:04.909492 15577 consensus_queue.cc:237] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } }
I20260925 10:58:04.913923 15091 catalog_manager.cc:5495] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32389eab881f4599b46dba9d30079bc1 (127.31.62.131). New cstate: current_term: 1 leader_uuid: "32389eab881f4599b46dba9d30079bc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } health_report { overall_health: UNKNOWN } } }
I20260925 10:58:04.950176 15568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.944203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41932 (local address 127.31.62.131:43497)
0925 10:58:04.944376 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:04.944381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:04.944460 (+ 79us) server_negotiation.cc:407] Connection header received
0925 10:58:04.944669 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:04.944673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:04.944750 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:04.944909 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:58:04.945258 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.945960 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.947672 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:04.948155 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:04.948265 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:04.948931 (+ 666us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:58:04.949529 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:58:04.949667 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:04.949844 (+ 177us) server_negotiation.cc:299] Negotiation successful
0925 10:58:04.949931 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 10:58:04.959887 15237 raft_consensus.cc:1261] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea [term 1 FOLLOWER]: Refusing update from remote peer 32389eab881f4599b46dba9d30079bc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:58:04.959894 15373 raft_consensus.cc:1261] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 1 FOLLOWER]: Refusing update from remote peer 32389eab881f4599b46dba9d30079bc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:58:04.960981 15580 consensus_queue.cc:1035] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:58:04.961441 15577 consensus_queue.cc:1035] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:58:04.978608 15584 mvcc.cc:205] Tried to move back new op lower bound from 7333207592784916480 to 7333207592593174528. Current Snapshot: MvccSnapshot[applied={T|T < 7333207592784916480}]
I20260925 10:58:04.984354 15092 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:58:04.984647 15092 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:58:04.988440 15092 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Sending DeleteTablet for 3 replicas of tablet e23008b3710f43c2b92c24ba2d02365c
I20260925 10:58:04.989504 15351 tablet_service.cc:1514] Processing DeleteTablet for tablet e23008b3710f43c2b92c24ba2d02365c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:58:04 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:50700
I20260925 10:58:04.989691 15486 tablet_service.cc:1514] Processing DeleteTablet for tablet e23008b3710f43c2b92c24ba2d02365c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:58:04 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:41922
I20260925 10:58:04.990406 15592 tablet_replica.cc:330] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268: stopping tablet replica
I20260925 10:58:04.990542 15217 tablet_service.cc:1514] Processing DeleteTablet for tablet e23008b3710f43c2b92c24ba2d02365c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:58:04 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:34902
I20260925 10:58:04.990862 15592 raft_consensus.cc:2229] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:58:04.991353 15592 raft_consensus.cc:2258] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:58:04.990715 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 15154
I20260925 10:58:04.993731 15592 ts_tablet_manager.cc:1905] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:58:05.003131 15594 tablet_replica.cc:330] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea: stopping tablet replica
I20260925 10:58:05.003839 15594 raft_consensus.cc:2229] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:58:05.004498 15594 raft_consensus.cc:2258] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:58:05.006139 15593 tablet_replica.cc:330] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1: stopping tablet replica
I20260925 10:58:05.006655 15593 raft_consensus.cc:2229] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:58:05.007370 15593 raft_consensus.cc:2258] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:58:05.010985 15593 ts_tablet_manager.cc:1905] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:58:05.011003 15594 ts_tablet_manager.cc:1905] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:58:05.012168 15592 ts_tablet_manager.cc:1918] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:58:05.012389 15592 log.cc:1196] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/e23008b3710f43c2b92c24ba2d02365c
I20260925 10:58:05.013021 15592 ts_tablet_manager.cc:1939] T e23008b3710f43c2b92c24ba2d02365c P 3334a8fea2ea46bbadf2f9c28cd5e268: Deleting consensus metadata
I20260925 10:58:05.015010 15079 catalog_manager.cc:4841] TS 3334a8fea2ea46bbadf2f9c28cd5e268 (127.31.62.130:35557): tablet e23008b3710f43c2b92c24ba2d02365c (table test-table [id=006f6f47c9ee4f11b4ff37e96e086a60]) successfully deleted
Sep 25 10:58:05 dist-test-slave-nx7s krb5kdc[15047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:58:05.017869 15594 ts_tablet_manager.cc:1918] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:58:05.017869 15593 ts_tablet_manager.cc:1918] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:58:05.018044 15593 log.cc:1196] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/e23008b3710f43c2b92c24ba2d02365c
I20260925 10:58:05.018172 15594 log.cc:1196] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/e23008b3710f43c2b92c24ba2d02365c
I20260925 10:58:05.018560 15593 ts_tablet_manager.cc:1939] T e23008b3710f43c2b92c24ba2d02365c P 32389eab881f4599b46dba9d30079bc1: Deleting consensus metadata
I20260925 10:58:05.018733 15594 ts_tablet_manager.cc:1939] T e23008b3710f43c2b92c24ba2d02365c P 0b1719c04ff84ab59df62b3e866578ea: Deleting consensus metadata
I20260925 10:58:05.020401 15076 catalog_manager.cc:4841] TS 0b1719c04ff84ab59df62b3e866578ea (127.31.62.129:44585): tablet e23008b3710f43c2b92c24ba2d02365c (table test-table [id=006f6f47c9ee4f11b4ff37e96e086a60]) successfully deleted
I20260925 10:58:05.020685 15078 catalog_manager.cc:4841] TS 32389eab881f4599b46dba9d30079bc1 (127.31.62.131:43497): tablet e23008b3710f43c2b92c24ba2d02365c (table test-table [id=006f6f47c9ee4f11b4ff37e96e086a60]) successfully deleted
I20260925 10:58:05.024096 15569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:04.993045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34906 (local address 127.31.62.129:44585)
0925 10:58:04.993227 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:04.993233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:05.005479 (+ 12246us) server_negotiation.cc:407] Connection header received
0925 10:58:05.005712 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:05.005719 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:05.005806 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:05.005898 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:05.007927 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:05.008704 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:05.010584 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:05.011250 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:05.011347 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:05.017128 (+ 5781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:05.017145 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:05.017169 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:05.017213 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:05.021125 (+ 3912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:05.021950 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:05.021959 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:05.021962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:05.022123 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:05.022525 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:05.022530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:05.022533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:05.022988 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:05.023283 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:05.023514 (+ 231us) server_negotiation.cc:299] Negotiation successful
0925 10:58:05.023731 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 10:58:05.024796 15277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:58:05.113739 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 15154
I20260925 10:58:05.135059 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 15288
Sep 25 10:58:05 dist-test-slave-nx7s krb5kdc[15047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:58:05.154747 15564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:05.135964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50708 (local address 127.31.62.130:35557)
0925 10:58:05.136169 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:05.136176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:05.136199 (+ 23us) server_negotiation.cc:407] Connection header received
0925 10:58:05.136392 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:05.136398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:05.136489 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:05.136604 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:05.138415 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:05.139571 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:05.141516 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:05.142184 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:05.142336 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:05.147975 (+ 5639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:05.147998 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:05.148018 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:05.148065 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:05.151156 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:05.152278 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:05.152287 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:05.152289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:05.152432 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:05.152960 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:05.152965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:05.152968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:05.153374 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:05.153628 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:05.154232 (+ 604us) server_negotiation.cc:299] Negotiation successful
0925 10:58:05.154439 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 10:58:05.155545 15413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:58:05.244199 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 15288
I20260925 10:58:05.265748 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 15423
Sep 25 10:58:05 dist-test-slave-nx7s krb5kdc[15047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:58:05.285789 15568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:05.266555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41934 (local address 127.31.62.131:43497)
0925 10:58:05.266714 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:05.266721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:05.266738 (+ 17us) server_negotiation.cc:407] Connection header received
0925 10:58:05.266903 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:05.266907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:05.267002 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:05.267088 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:05.268987 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:05.269828 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:05.271688 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:05.272289 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:05.272388 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:05.277916 (+ 5528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:05.277938 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:05.277960 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:05.278010 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:05.281998 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:05.282823 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:05.282833 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:05.282836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:05.283061 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:05.283639 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:05.283649 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:05.283653 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:05.284142 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:05.284620 (+ 478us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:05.284803 (+ 183us) server_negotiation.cc:299] Negotiation successful
0925 10:58:05.285366 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 10:58:05.286543 15547 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:58:05.408267 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 15423
I20260925 10:58:05.423412 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 15060
I20260925 10:58:05.424489 15122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:58:05.534065 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 15060
I20260925 10:58:25.551257 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:34111
--webserver_interface=127.31.62.190
--webserver_port=37889
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:34111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:58:25.882323 15600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:58:25.882746 15600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:58:25.882892 15600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:58:25.890431 15600 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 10:58:25.890537 15600 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 10:58:25.890578 15600 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 10:58:25.890645 15600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:58:25.890718 15600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:58:25.890810 15600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:58:25.890868 15600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:58:25.890940 15600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:58:25.901013 15600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:34111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:34111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=37889
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:58:25.902737 15600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:58:25.904193 15600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:58:25.926164 15607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:25.926200 15609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:25.926384 15606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:58:25.927135 15600 server_base.cc:1034] running on GCE node
I20260925 10:58:25.928050 15600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:58:25.929795 15600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:58:25.931041 15600 hybrid_clock.cc:648] HybridClock initialized: now 1790333905930963 us; error 50 us; skew 500 ppm
I20260925 10:58:25.931454 15600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:58:25 dist-test-slave-nx7s krb5kdc[15047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333905, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:58:25.935257 15600 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:58:25.937033 15600 webserver.cc:460] Webserver started at http://127.31.62.190:37889/ using document root <none> and password file <none>
I20260925 10:58:25.937819 15600 fs_manager.cc:359] Metadata directory not provided
I20260925 10:58:25.937951 15600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:58:25.944140 15600 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 10:58:25.947254 15616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:25.949054 15600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:58:25.949231 15600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5"
format_stamp: "Formatted at 2026-09-25 10:58:03 on dist-test-slave-nx7s"
server_key: "c23fd8b3a87778a20fbd5777e78f3d1f"
server_key_iv: "9822d4d372faf87b1bbbd4235598c1a6"
server_key_version: "encryptionkey@0"
I20260925 10:58:25.950292 15600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:58:25.970669 15600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:58:25.975277 15600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:58:25.975577 15600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:58:25.994205 15600 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:34111
I20260925 10:58:25.994256 15668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:34111 every 8 connection(s)
I20260925 10:58:25.996222 15600 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:58:26.003522 15669 sys_catalog.cc:263] Verifying existing consensus state
I20260925 10:58:26.004886 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 15600
I20260925 10:58:26.005874 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:44585
--local_ip_for_outbound_sockets=127.31.62.129
--tserver_master_addrs=127.31.62.190:34111
--webserver_port=46689
--webserver_interface=127.31.62.129
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:58:26.008109 15669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Bootstrap starting.
I20260925 10:58:26.030149 15669 log.cc:824] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Log is configured to *not* fsync() on all Append() calls
I20260925 10:58:26.043009 15669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 10:58:26.043551 15669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Bootstrap complete.
I20260925 10:58:26.052754 15669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } }
I20260925 10:58:26.053664 15669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa5d569e58449a3ad573f2cf3d3ffe5, State: Initialized, Role: FOLLOWER
I20260925 10:58:26.054332 15669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } }
I20260925 10:58:26.054569 15669 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:58:26.054687 15669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:58:26.054883 15669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 1 FOLLOWER]: Advancing to term 2
I20260925 10:58:26.058909 15669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } }
I20260925 10:58:26.059450 15669 leader_election.cc:302] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1aa5d569e58449a3ad573f2cf3d3ffe5; no voters:
I20260925 10:58:26.060050 15669 leader_election.cc:288] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 10:58:26.060220 15673 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 10:58:26.061309 15673 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 2 LEADER]: Becoming Leader. State: Replica: 1aa5d569e58449a3ad573f2cf3d3ffe5, State: Running, Role: LEADER
I20260925 10:58:26.061893 15673 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } }
I20260925 10:58:26.062384 15669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:58:26.064342 15674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } } }
I20260925 10:58:26.064738 15674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:58:26.065510 15675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aa5d569e58449a3ad573f2cf3d3ffe5. Latest consensus state: current_term: 2 leader_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } } }
I20260925 10:58:26.065884 15675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:58:26.068552 15678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:58:26.077266 15678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:58:26.078859 15678 catalog_manager.cc:1259] Loaded cluster ID: 76e51fd4b12043608d673ea1428510ee
I20260925 10:58:26.079063 15678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:58:26.083468 15678 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:58:26.085136 15678 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Loaded TSK: 0
I20260925 10:58:26.123947 15690 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 10:58:26.143733 15678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Generated new TSK 1
I20260925 10:58:26.144164 15678 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 10:58:26.373077 15671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:58:26.373397 15671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:58:26.373533 15671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:58:26.381001 15671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:58:26.381127 15671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:58:26.381278 15671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:58:26.391806 15671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:44585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=46689
--tserver_master_addrs=127.31.62.190:34111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:58:26.393239 15671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:58:26.394665 15671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:58:26.409730 15696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:26.409725 15697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:58:26.410435 15671 server_base.cc:1034] running on GCE node
W20260925 10:58:26.410522 15699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:58:26.411340 15671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:58:26.412619 15671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:58:26.413875 15671 hybrid_clock.cc:648] HybridClock initialized: now 1790333906413780 us; error 88 us; skew 500 ppm
I20260925 10:58:26.414283 15671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:58:26 dist-test-slave-nx7s krb5kdc[15047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333906, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:58:26.418759 15671 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:58:26.420745 15671 webserver.cc:460] Webserver started at http://127.31.62.129:46689/ using document root <none> and password file <none>
I20260925 10:58:26.421795 15671 fs_manager.cc:359] Metadata directory not provided
I20260925 10:58:26.421977 15671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:58:26.429801 15671 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260925 10:58:26.433626 15706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:26.435472 15671 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 10:58:26.435688 15671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "0b1719c04ff84ab59df62b3e866578ea"
format_stamp: "Formatted at 2026-09-25 10:58:03 on dist-test-slave-nx7s"
server_key: "bbf81a67656c5fa9c5534108de43e8cc"
server_key_iv: "3c1aa285889b4c2a189d39d3b3bba724"
server_key_version: "encryptionkey@0"
I20260925 10:58:26.437166 15671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:58:26.490965 15671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:58:26.495450 15671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:58:26.495728 15671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:58:26.496930 15671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:58:26.498445 15671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:58:26.498548 15671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:26.498631 15671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:58:26.498685 15671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:26.530365 15671 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:44585
I20260925 10:58:26.530426 15819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:44585 every 8 connection(s)
I20260925 10:58:26.531971 15671 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:58:26.534626 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 15671
I20260925 10:58:26.535362 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:35557
--local_ip_for_outbound_sockets=127.31.62.130
--tserver_master_addrs=127.31.62.190:34111
--webserver_port=45021
--webserver_interface=127.31.62.130
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 10:58:26 dist-test-slave-nx7s krb5kdc[15047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333906, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:58:26.556584 15822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:26.534891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:57431 (local address 127.31.62.190:34111)
0925 10:58:26.535453 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:26.535469 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:26.536241 (+ 772us) server_negotiation.cc:407] Connection header received
0925 10:58:26.538495 (+ 2254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:26.538510 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:26.538875 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:26.539237 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:26.540870 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:26.542197 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:26.543861 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:26.544601 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:26.544777 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:26.548594 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:26.548602 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:26.548621 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:26.548662 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:26.551631 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:26.552445 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:26.552452 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:26.552459 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:26.552635 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:26.553202 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:26.553206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:26.553208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:26.553769 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:26.553957 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:26.554876 (+ 919us) server_negotiation.cc:299] Negotiation successful
0925 10:58:26.555276 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":233,"threads_started":1}
I20260925 10:58:26.556584 15823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:26.535265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:57431)
0925 10:58:26.536055 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:26.536094 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:26.537920 (+ 1826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:26.539406 (+ 1486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:26.539416 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:26.540058 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:58:26.540641 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:26.540660 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:26.542368 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:26.542372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:26.543713 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:26.543728 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:26.544838 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:26.544880 (+ 42us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:26.545149 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:26.545829 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:58:26.545853 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:58:26.548283 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:58:26.551802 (+ 3519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:58:26.551813 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:58:26.551831 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:58:26.552291 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:58:26.552753 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:58:26.552758 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:58:26.552762 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:58:26.553087 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:58:26.553917 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:58:26.553926 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:58:26.554297 (+ 371us) client_negotiation.cc:769] Sending connection context
0925 10:58:26.554604 (+ 307us) client_negotiation.cc:240] Negotiation successful
0925 10:58:26.555215 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":316,"threads_started":1}
I20260925 10:58:26.560099 15820 heartbeater.cc:347] Connected to a master server at 127.31.62.190:34111
I20260925 10:58:26.560457 15820 heartbeater.cc:464] Registering TS with master...
I20260925 10:58:26.561372 15820 heartbeater.cc:511] Master 127.31.62.190:34111 requested a full tablet report, sending...
I20260925 10:58:26.564126 15633 ts_manager.cc:194] Registered new tserver with Master: 0b1719c04ff84ab59df62b3e866578ea (127.31.62.129:44585)
I20260925 10:58:26.565713 15633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:57431
W20260925 10:58:26.870141 15824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:58:26.870435 15824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:58:26.870555 15824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:58:26.877941 15824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:58:26.878067 15824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:58:26.878218 15824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:58:26.888749 15824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:35557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=45021
--tserver_master_addrs=127.31.62.190:34111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:58:26.890117 15824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:58:26.891433 15824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:58:26.904718 15829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:26.904685 15832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:26.904753 15830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:58:26.905474 15824 server_base.cc:1034] running on GCE node
I20260925 10:58:26.906152 15824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:58:26.907164 15824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:58:26.908406 15824 hybrid_clock.cc:648] HybridClock initialized: now 1790333906908341 us; error 49 us; skew 500 ppm
I20260925 10:58:26.908744 15824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:58:26 dist-test-slave-nx7s krb5kdc[15047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333906, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:58:26.912068 15824 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:58:26.913740 15824 webserver.cc:460] Webserver started at http://127.31.62.130:45021/ using document root <none> and password file <none>
I20260925 10:58:26.914459 15824 fs_manager.cc:359] Metadata directory not provided
I20260925 10:58:26.914597 15824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:58:26.920416 15824 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 10:58:26.923511 15839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:26.925221 15824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260925 10:58:26.925388 15824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "3334a8fea2ea46bbadf2f9c28cd5e268"
format_stamp: "Formatted at 2026-09-25 10:58:04 on dist-test-slave-nx7s"
server_key: "1ab346a62880dec4b7f75f55e816c0fd"
server_key_iv: "62e19caf289c28aae032315ea65244f1"
server_key_version: "encryptionkey@0"
I20260925 10:58:26.926388 15824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:58:26.947712 15824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:58:26.952421 15824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:58:26.952720 15824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:58:26.953763 15824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:58:26.955297 15824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:58:26.955406 15824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:26.955519 15824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:58:26.955587 15824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:26.987111 15824 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:35557
I20260925 10:58:26.987180 15952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:35557 every 8 connection(s)
I20260925 10:58:26.988595 15824 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:58:26.989241 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 15824
I20260925 10:58:26.990036 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:43497
--local_ip_for_outbound_sockets=127.31.62.131
--tserver_master_addrs=127.31.62.190:34111
--webserver_port=43259
--webserver_interface=127.31.62.131
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 10:58:27 dist-test-slave-nx7s krb5kdc[15047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333906, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:58:27.015630 15822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:26.991709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:54875 (local address 127.31.62.190:34111)
0925 10:58:26.991946 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:26.991953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:26.993456 (+ 1503us) server_negotiation.cc:407] Connection header received
0925 10:58:26.996146 (+ 2690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:26.996153 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:26.996292 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:26.996418 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:26.998652 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:26.999728 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.002134 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.002843 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.002988 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.007839 (+ 4851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:27.007847 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:27.007852 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:27.007905 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:27.011671 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:27.012914 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:27.012921 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:27.012924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:27.013112 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:27.013680 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:27.013684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:27.013686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:27.013989 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:27.014159 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:27.015152 (+ 993us) server_negotiation.cc:299] Negotiation successful
0925 10:58:27.015370 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 10:58:27.017367 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:26.992362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:54875)
0925 10:58:26.993254 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:26.993310 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:26.995748 (+ 2438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:26.996715 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:26.996728 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:26.997512 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:58:26.998378 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:26.998421 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:26.999960 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:26.999966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:27.001938 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:27.001954 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.003119 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.003125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:27.003415 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.004299 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:58:27.004334 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:58:27.007508 (+ 3174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:58:27.011920 (+ 4412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:58:27.011933 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:58:27.011955 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:58:27.012658 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:58:27.013240 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:58:27.013246 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:58:27.013249 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:58:27.013542 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:58:27.014164 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:58:27.014173 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:58:27.014697 (+ 524us) client_negotiation.cc:769] Sending connection context
0925 10:58:27.015039 (+ 342us) client_negotiation.cc:240] Negotiation successful
0925 10:58:27.015708 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":339,"threads_started":1}
I20260925 10:58:27.019002 15953 heartbeater.cc:347] Connected to a master server at 127.31.62.190:34111
I20260925 10:58:27.019369 15953 heartbeater.cc:464] Registering TS with master...
I20260925 10:58:27.020277 15953 heartbeater.cc:511] Master 127.31.62.190:34111 requested a full tablet report, sending...
I20260925 10:58:27.022179 15633 ts_manager.cc:194] Registered new tserver with Master: 3334a8fea2ea46bbadf2f9c28cd5e268 (127.31.62.130:35557)
I20260925 10:58:27.023043 15633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:54875
W20260925 10:58:27.331102 15956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:58:27.331413 15956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:58:27.331540 15956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:58:27.339401 15956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:58:27.339540 15956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:58:27.339694 15956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:58:27.350188 15956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:43497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=43259
--tserver_master_addrs=127.31.62.190:34111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:58:27.351552 15956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:58:27.352911 15956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:58:27.370229 15961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:27.370270 15962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:27.370290 15964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:58:27.370898 15956 server_base.cc:1034] running on GCE node
I20260925 10:58:27.371510 15956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:58:27.372539 15956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:58:27.373742 15956 hybrid_clock.cc:648] HybridClock initialized: now 1790333907373690 us; error 38 us; skew 500 ppm
I20260925 10:58:27.374163 15956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:58:27 dist-test-slave-nx7s krb5kdc[15047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333907, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:58:27.377328 15956 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:58:27.378901 15956 webserver.cc:460] Webserver started at http://127.31.62.131:43259/ using document root <none> and password file <none>
I20260925 10:58:27.379602 15956 fs_manager.cc:359] Metadata directory not provided
I20260925 10:58:27.379734 15956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:58:27.385458 15956 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 10:58:27.388545 15971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:27.390041 15956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:58:27.390223 15956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "32389eab881f4599b46dba9d30079bc1"
format_stamp: "Formatted at 2026-09-25 10:58:04 on dist-test-slave-nx7s"
server_key: "dfa42167be4081a996b9041a7ce50ac1"
server_key_iv: "bebe7e8b8c7e5ff28f9714f3fcf26bb8"
server_key_version: "encryptionkey@0"
I20260925 10:58:27.391225 15956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:58:27.414819 15956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:58:27.419546 15956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:58:27.419850 15956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:58:27.420979 15956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:58:27.422495 15956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:58:27.422598 15956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:27.422740 15956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:58:27.422827 15956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:27.456175 15956 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:43497
I20260925 10:58:27.456235 16084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:43497 every 8 connection(s)
I20260925 10:58:27.457809 15956 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:58:27.458525 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 15956
Sep 25 10:58:27 dist-test-slave-nx7s krb5kdc[15047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333907, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:58:27.474126 15822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:27.459726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56228 (local address 127.31.62.190:34111)
0925 10:58:27.459934 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:27.459942 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:27.460376 (+ 434us) server_negotiation.cc:407] Connection header received
0925 10:58:27.460574 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:27.460580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:27.460671 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:27.460785 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:27.462449 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.463354 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.464787 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.465430 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.465534 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.468193 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:27.468198 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:27.468201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:27.468235 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:27.470949 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:27.471738 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:27.471743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:27.471747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:27.471888 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:27.472394 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:27.472398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:27.472401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:27.472726 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:27.473040 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:27.473545 (+ 505us) server_negotiation.cc:299] Negotiation successful
0925 10:58:27.473780 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 10:58:27.479959 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:27.461049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:44773 (local address 127.31.62.190:34111)
0925 10:58:27.461515 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:27.461520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:27.462911 (+ 1391us) server_negotiation.cc:407] Connection header received
0925 10:58:27.465039 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:27.465043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:27.465124 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:27.465232 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:27.467030 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.467751 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.469389 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.469829 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.469950 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.474070 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:27.474075 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:27.474078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:27.474107 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:27.476946 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:27.477815 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:27.477819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:27.477822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:27.478002 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:27.478419 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:27.478421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:27.478423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:27.478705 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:27.478822 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:27.479500 (+ 678us) server_negotiation.cc:299] Negotiation successful
0925 10:58:27.479666 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":224,"threads_started":1}
I20260925 10:58:27.481038 16089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:27.461826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:44773)
0925 10:58:27.462745 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:27.462793 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:27.464739 (+ 1946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:27.465511 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:27.465520 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:27.466259 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:58:27.466838 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:27.466855 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.467910 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.467914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:27.469231 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:27.469244 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.470071 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.470079 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:27.470328 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.470988 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:58:27.471018 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:58:27.473760 (+ 2742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:58:27.477131 (+ 3371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:58:27.477141 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:58:27.477162 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:58:27.477666 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:58:27.478114 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:58:27.478117 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:58:27.478120 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:58:27.478327 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:58:27.478813 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:58:27.478819 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:58:27.479174 (+ 355us) client_negotiation.cc:769] Sending connection context
0925 10:58:27.479409 (+ 235us) client_negotiation.cc:240] Negotiation successful
0925 10:58:27.479829 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":385,"threads_started":1}
I20260925 10:58:27.482311 16085 heartbeater.cc:347] Connected to a master server at 127.31.62.190:34111
I20260925 10:58:27.482645 16085 heartbeater.cc:464] Registering TS with master...
I20260925 10:58:27.483326 16085 heartbeater.cc:511] Master 127.31.62.190:34111 requested a full tablet report, sending...
I20260925 10:58:27.484769 15633 ts_manager.cc:194] Registered new tserver with Master: 32389eab881f4599b46dba9d30079bc1 (127.31.62.131:43497)
I20260925 10:58:27.485685 15633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:44773
I20260925 10:58:27.489531 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:58:27.496059 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:27.490502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56240 (local address 127.31.62.190:34111)
0925 10:58:27.490687 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:27.490694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:27.491054 (+ 360us) server_negotiation.cc:407] Connection header received
0925 10:58:27.491219 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:27.491223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:27.491384 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:27.491525 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:58:27.491923 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.492665 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.494152 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.494589 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.494694 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.495230 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:58:27.495437 (+ 207us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0925 10:58:27.495751 (+ 314us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56240: token expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260925 10:58:27.496197 16088 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56240: token expired
W20260925 10:58:27.496719 15558 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.62.190:34111)
I20260925 10:58:27.509356 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:27.497207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56250 (local address 127.31.62.190:34111)
0925 10:58:27.497346 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:27.497349 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:27.497363 (+ 14us) server_negotiation.cc:407] Connection header received
0925 10:58:27.497411 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:27.497414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:27.497476 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:27.497551 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:27.498977 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.499672 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.501143 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.501533 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.501624 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.504252 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:27.504256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:27.504259 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:27.504292 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:27.506812 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:27.507569 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:27.507573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:27.507575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:27.507696 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:27.508094 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:27.508097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:27.508099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:27.508404 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:27.508596 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:27.508937 (+ 341us) server_negotiation.cc:299] Negotiation successful
0925 10:58:27.509112 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 10:58:27.513250 15633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:58:27.521180 15633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:58:27.567519 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:27.534277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44586 (local address 127.31.62.131:43497)
0925 10:58:27.557054 (+ 22777us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:27.557064 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:27.558217 (+ 1153us) server_negotiation.cc:407] Connection header received
0925 10:58:27.558352 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:27.558362 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:27.558683 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:27.558918 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:58:27.559859 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.561599 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.564680 (+ 3081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.566398 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.566509 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.566736 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:27.567043 (+ 307us) server_negotiation.cc:299] Negotiation successful
0925 10:58:27.567195 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22556,"thread_start_us":22413,"threads_started":1}
I20260925 10:58:27.567864 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:27.533630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:43497 (local address 127.0.0.1:44586)
0925 10:58:27.557888 (+ 24258us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:27.557909 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:27.558073 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:27.559120 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:27.559129 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:27.559170 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:58:27.559650 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:27.559668 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.561768 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.561774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:27.564439 (+ 2665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:27.564455 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.566600 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.566606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:27.566810 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.566858 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 10:58:27.566978 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 10:58:27.567171 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24105,"mutex_wait_us":94,"thread_start_us":23921,"threads_started":1}
I20260925 10:58:27.567919 16099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:27.533494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:44585 (local address 127.0.0.1:43350)
0925 10:58:27.555677 (+ 22183us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:27.555735 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:27.556001 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:27.560328 (+ 4327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:27.560333 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:27.560358 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:58:27.560773 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:27.560786 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.562923 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.562928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:27.564821 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:27.564834 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.566904 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.566911 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:27.567073 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.567107 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 10:58:27.567180 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 10:58:27.567316 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21823,"mutex_wait_us":115,"thread_start_us":26436,"threads_started":1}
I20260925 10:58:27.568307 16097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:27.533750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.31.62.129:44585)
0925 10:58:27.559715 (+ 25965us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:27.559725 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:27.559757 (+ 32us) server_negotiation.cc:407] Connection header received
0925 10:58:27.559861 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:27.559871 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:27.560135 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:27.560316 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:58:27.560944 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.562769 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.565027 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.566387 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.566513 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.566747 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:27.567782 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0925 10:58:27.567955 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25779,"thread_start_us":25605,"threads_started":1}
I20260925 10:58:27.568781 16095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:27.537678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:35557 (local address 127.0.0.1:42496)
0925 10:58:27.560190 (+ 22512us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:27.560209 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:27.560500 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:27.561378 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:27.561382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:27.561405 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:58:27.561760 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:27.561771 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.563708 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.563713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:27.565967 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:27.565981 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.567992 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.567998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:27.568169 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.568210 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 10:58:27.568294 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 10:58:27.568441 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22390,"mutex_wait_us":95,"spinlock_wait_cycles":11648,"thread_start_us":17476,"threads_started":1}
I20260925 10:58:27.569748 15820 heartbeater.cc:503] Master 127.31.62.190:34111 was elected leader, sending a full tablet report...
I20260925 10:58:27.572227 15755 tablet_service.cc:1467] Processing CreateTablet for tablet 827e5c3b47084c12b19844ccfd005999 (DEFAULT_TABLE table=test-table [id=97513378bf2a4b998e84e6edcc9c19db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:58:27.572222 16020 tablet_service.cc:1467] Processing CreateTablet for tablet 827e5c3b47084c12b19844ccfd005999 (DEFAULT_TABLE table=test-table [id=97513378bf2a4b998e84e6edcc9c19db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:58:27.573792 15755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827e5c3b47084c12b19844ccfd005999. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:58:27.573787 16020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827e5c3b47084c12b19844ccfd005999. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:58:27.574055 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:27.538273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42496 (local address 127.31.62.130:35557)
0925 10:58:27.556504 (+ 18231us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:27.556516 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:27.560308 (+ 3792us) server_negotiation.cc:407] Connection header received
0925 10:58:27.560706 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:27.560714 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:27.560967 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:27.561136 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:58:27.561949 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.563541 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.566167 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.567533 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.567663 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.567843 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:27.569019 (+ 1176us) server_negotiation.cc:299] Negotiation successful
0925 10:58:27.569152 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17981,"thread_start_us":18872,"threads_started":1}
I20260925 10:58:27.577399 15888 tablet_service.cc:1467] Processing CreateTablet for tablet 827e5c3b47084c12b19844ccfd005999 (DEFAULT_TABLE table=test-table [id=97513378bf2a4b998e84e6edcc9c19db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:58:27.578966 15888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827e5c3b47084c12b19844ccfd005999. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:58:27.590653 16103 tablet_bootstrap.cc:492] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268: Bootstrap starting.
I20260925 10:58:27.595429 16103 tablet_bootstrap.cc:654] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268: Neither blocks nor log segments found. Creating new log.
I20260925 10:58:27.595686 16101 tablet_bootstrap.cc:492] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea: Bootstrap starting.
I20260925 10:58:27.596664 16103 log.cc:824] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268: Log is configured to *not* fsync() on all Append() calls
I20260925 10:58:27.599807 16101 tablet_bootstrap.cc:654] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea: Neither blocks nor log segments found. Creating new log.
I20260925 10:58:27.599987 16103 tablet_bootstrap.cc:492] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268: No bootstrap required, opened a new log
I20260925 10:58:27.600255 16103 ts_tablet_manager.cc:1397] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260925 10:58:27.601313 16101 log.cc:824] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea: Log is configured to *not* fsync() on all Append() calls
I20260925 10:58:27.605917 16102 tablet_bootstrap.cc:492] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1: Bootstrap starting.
I20260925 10:58:27.609803 16102 tablet_bootstrap.cc:654] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1: Neither blocks nor log segments found. Creating new log.
I20260925 10:58:27.609473 16103 raft_consensus.cc:348] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } }
I20260925 10:58:27.609964 16103 raft_consensus.cc:374] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:58:27.610071 16103 raft_consensus.cc:729] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3334a8fea2ea46bbadf2f9c28cd5e268, State: Initialized, Role: FOLLOWER
I20260925 10:58:27.610442 16101 tablet_bootstrap.cc:492] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea: No bootstrap required, opened a new log
I20260925 10:58:27.610711 16101 ts_tablet_manager.cc:1397] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea: Time spent bootstrapping tablet: real 0.015s user 0.013s sys 0.000s
I20260925 10:58:27.610746 16103 consensus_queue.cc:260] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } }
I20260925 10:58:27.611085 16102 log.cc:824] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1: Log is configured to *not* fsync() on all Append() calls
I20260925 10:58:27.617676 15953 heartbeater.cc:503] Master 127.31.62.190:34111 was elected leader, sending a full tablet report...
I20260925 10:58:27.618489 16103 ts_tablet_manager.cc:1428] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268: Time spent starting tablet: real 0.018s user 0.019s sys 0.000s
I20260925 10:58:27.618803 16101 raft_consensus.cc:348] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } }
I20260925 10:58:27.619293 16101 raft_consensus.cc:374] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:58:27.619405 16101 raft_consensus.cc:729] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1719c04ff84ab59df62b3e866578ea, State: Initialized, Role: FOLLOWER
I20260925 10:58:27.620071 16101 consensus_queue.cc:260] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } }
I20260925 10:58:27.621140 16102 tablet_bootstrap.cc:492] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1: No bootstrap required, opened a new log
I20260925 10:58:27.621435 16102 ts_tablet_manager.cc:1397] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1: Time spent bootstrapping tablet: real 0.016s user 0.014s sys 0.000s
I20260925 10:58:27.621943 16101 ts_tablet_manager.cc:1428] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea: Time spent starting tablet: real 0.011s user 0.004s sys 0.008s
I20260925 10:58:27.630327 16102 raft_consensus.cc:348] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } }
I20260925 10:58:27.630745 16102 raft_consensus.cc:374] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:58:27.630854 16102 raft_consensus.cc:729] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32389eab881f4599b46dba9d30079bc1, State: Initialized, Role: FOLLOWER
I20260925 10:58:27.631464 16102 consensus_queue.cc:260] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } }
I20260925 10:58:27.632530 16085 heartbeater.cc:503] Master 127.31.62.190:34111 was elected leader, sending a full tablet report...
I20260925 10:58:27.633037 16102 ts_tablet_manager.cc:1428] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
W20260925 10:58:27.709786 16086 tablet.cc:2369] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:58:27.740839 15954 tablet.cc:2369] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:58:27.784475 15821 tablet.cc:2369] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:58:27.968999 16106 raft_consensus.cc:482] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:58:27.969280 16106 raft_consensus.cc:504] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } }
I20260925 10:58:27.971251 16106 leader_election.cc:288] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b1719c04ff84ab59df62b3e866578ea (127.31.62.129:44585), 32389eab881f4599b46dba9d30079bc1 (127.31.62.131:43497)
I20260925 10:58:27.980361 16097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:27.971838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:33555 (local address 127.31.62.129:44585)
0925 10:58:27.972041 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:27.972049 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:27.973134 (+ 1085us) server_negotiation.cc:407] Connection header received
0925 10:58:27.973301 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:27.973305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:27.973366 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:27.973441 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:58:27.974023 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.975058 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.977800 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.979096 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.979222 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.979301 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:27.979927 (+ 626us) server_negotiation.cc:299] Negotiation successful
0925 10:58:27.980092 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 10:58:27.980690 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:27.971784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:51317 (local address 127.31.62.131:43497)
0925 10:58:27.972057 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:27.972064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:27.972974 (+ 910us) server_negotiation.cc:407] Connection header received
0925 10:58:27.973288 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:27.973295 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:27.973391 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:27.973455 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:58:27.974424 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.975406 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.977782 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.979068 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.979222 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.979301 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:27.980306 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0925 10:58:27.980459 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260925 10:58:27.981253 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:27.972246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:44585 (local address 127.31.62.130:33555)
0925 10:58:27.973027 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:27.973042 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:27.973167 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:27.973474 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:27.973478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:27.973502 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:58:27.973861 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:27.973871 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.975229 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.975234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:27.977584 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:27.977601 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.979524 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.979531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:27.979707 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.979738 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 10:58:27.980650 (+ 912us) client_negotiation.cc:240] Negotiation successful
0925 10:58:27.980796 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":4,"thread_start_us":441,"threads_started":1}
I20260925 10:58:27.981515 16110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:27.971692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:43497 (local address 127.31.62.130:51317)
0925 10:58:27.972828 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:27.972873 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 10:58:27.973112 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:58:27.973735 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:58:27.973741 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:58:27.973770 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:58:27.974226 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:27.974238 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.975562 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.975567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:27.977623 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:58:27.977632 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:58:27.979930 (+ 2298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:58:27.979935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:58:27.980131 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:27.980162 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 10:58:27.980961 (+ 799us) client_negotiation.cc:240] Negotiation successful
0925 10:58:27.981106 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":950,"mutex_wait_us":2,"thread_start_us":553,"threads_started":1}
I20260925 10:58:27.982556 16040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827e5c3b47084c12b19844ccfd005999" candidate_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32389eab881f4599b46dba9d30079bc1" is_pre_election: true
I20260925 10:58:27.982575 15775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827e5c3b47084c12b19844ccfd005999" candidate_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1719c04ff84ab59df62b3e866578ea" is_pre_election: true
I20260925 10:58:27.983098 16040 raft_consensus.cc:2454] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3334a8fea2ea46bbadf2f9c28cd5e268 in term 0.
I20260925 10:58:27.983099 15775 raft_consensus.cc:2454] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3334a8fea2ea46bbadf2f9c28cd5e268 in term 0.
I20260925 10:58:27.983793 15842 leader_election.cc:302] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32389eab881f4599b46dba9d30079bc1, 3334a8fea2ea46bbadf2f9c28cd5e268; no voters:
I20260925 10:58:27.984195 16106 raft_consensus.cc:2781] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:58:27.984323 16106 raft_consensus.cc:482] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:58:27.984395 16106 raft_consensus.cc:3037] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:58:27.986711 16106 raft_consensus.cc:504] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } }
I20260925 10:58:27.987399 16106 leader_election.cc:288] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [CANDIDATE]: Term 1 election: Requested vote from peers 0b1719c04ff84ab59df62b3e866578ea (127.31.62.129:44585), 32389eab881f4599b46dba9d30079bc1 (127.31.62.131:43497)
I20260925 10:58:27.988003 16040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827e5c3b47084c12b19844ccfd005999" candidate_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32389eab881f4599b46dba9d30079bc1"
I20260925 10:58:27.988022 15775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827e5c3b47084c12b19844ccfd005999" candidate_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1719c04ff84ab59df62b3e866578ea"
I20260925 10:58:27.988201 16040 raft_consensus.cc:3037] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:58:27.988245 15775 raft_consensus.cc:3037] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:58:27.990464 16040 raft_consensus.cc:2454] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3334a8fea2ea46bbadf2f9c28cd5e268 in term 1.
I20260925 10:58:27.990948 15842 leader_election.cc:302] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32389eab881f4599b46dba9d30079bc1, 3334a8fea2ea46bbadf2f9c28cd5e268; no voters:
I20260925 10:58:27.991317 15775 raft_consensus.cc:2454] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3334a8fea2ea46bbadf2f9c28cd5e268 in term 1.
I20260925 10:58:27.991647 16106 raft_consensus.cc:2781] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:58:27.993930 16106 raft_consensus.cc:686] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 1 LEADER]: Becoming Leader. State: Replica: 3334a8fea2ea46bbadf2f9c28cd5e268, State: Running, Role: LEADER
I20260925 10:58:27.994724 16106 consensus_queue.cc:237] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } } peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } }
I20260925 10:58:27.999948 15633 catalog_manager.cc:5495] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3334a8fea2ea46bbadf2f9c28cd5e268 (127.31.62.130). New cstate: current_term: 1 leader_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3334a8fea2ea46bbadf2f9c28cd5e268" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35557 } health_report { overall_health: HEALTHY } } }
I20260925 10:58:28.029850 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:28.023879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42506 (local address 127.31.62.130:35557)
0925 10:58:28.024060 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:28.024065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:28.024383 (+ 318us) server_negotiation.cc:407] Connection header received
0925 10:58:28.024524 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:28.024529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:28.024658 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:28.024817 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:58:28.025299 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:28.026148 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:28.027531 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:28.028035 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:28.028158 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:28.028668 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:58:28.029305 (+ 637us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:58:28.029421 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:28.029555 (+ 134us) server_negotiation.cc:299] Negotiation successful
0925 10:58:28.029632 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 10:58:28.038061 16040 raft_consensus.cc:1261] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1 [term 1 FOLLOWER]: Refusing update from remote peer 3334a8fea2ea46bbadf2f9c28cd5e268: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:58:28.038061 15775 raft_consensus.cc:1261] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea [term 1 FOLLOWER]: Refusing update from remote peer 3334a8fea2ea46bbadf2f9c28cd5e268: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:58:28.039075 16112 consensus_queue.cc:1035] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32389eab881f4599b46dba9d30079bc1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 43497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:58:28.039427 16106 consensus_queue.cc:1035] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1719c04ff84ab59df62b3e866578ea" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 44585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:58:28.048637 16116 mvcc.cc:205] Tried to move back new op lower bound from 7333207687314223104 to 7333207687151128576. Current Snapshot: MvccSnapshot[applied={T|T < 7333207687314223104}]
I20260925 10:58:28.069051 15632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:58:28.069388 15632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:58:28.073163 15632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Sending DeleteTablet for 3 replicas of tablet 827e5c3b47084c12b19844ccfd005999
I20260925 10:58:28.074453 15755 tablet_service.cc:1514] Processing DeleteTablet for tablet 827e5c3b47084c12b19844ccfd005999 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:58:28 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:43350
I20260925 10:58:28.074445 16020 tablet_service.cc:1514] Processing DeleteTablet for tablet 827e5c3b47084c12b19844ccfd005999 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:58:28 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:44586
I20260925 10:58:28.074548 15888 tablet_service.cc:1514] Processing DeleteTablet for tablet 827e5c3b47084c12b19844ccfd005999 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:58:28 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:42496
I20260925 10:58:28.075881 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 15671
I20260925 10:58:28.087007 16125 tablet_replica.cc:330] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea: stopping tablet replica
I20260925 10:58:28.087754 16127 tablet_replica.cc:330] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268: stopping tablet replica
I20260925 10:58:28.088457 16127 raft_consensus.cc:2229] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:58:28.089092 16125 raft_consensus.cc:2229] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:58:28.090052 16127 raft_consensus.cc:2258] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:58:28.090677 16125 raft_consensus.cc:2258] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:58:28.091228 16126 tablet_replica.cc:330] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1: stopping tablet replica
I20260925 10:58:28.091647 16126 raft_consensus.cc:2229] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:58:28.092130 16126 raft_consensus.cc:2258] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:58:28.094094 16127 ts_tablet_manager.cc:1905] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:58:28.094087 16126 ts_tablet_manager.cc:1905] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:58:28.095022 16125 ts_tablet_manager.cc:1905] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:58:28.100963 16126 ts_tablet_manager.cc:1918] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:58:28.101155 16126 log.cc:1196] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/827e5c3b47084c12b19844ccfd005999
I20260925 10:58:28.101678 16126 ts_tablet_manager.cc:1939] T 827e5c3b47084c12b19844ccfd005999 P 32389eab881f4599b46dba9d30079bc1: Deleting consensus metadata
I20260925 10:58:28.103600 15619 catalog_manager.cc:4841] TS 32389eab881f4599b46dba9d30079bc1 (127.31.62.131:43497): tablet 827e5c3b47084c12b19844ccfd005999 (table test-table [id=97513378bf2a4b998e84e6edcc9c19db]) successfully deleted
I20260925 10:58:28.106690 16127 ts_tablet_manager.cc:1918] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:58:28.106859 16127 log.cc:1196] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/827e5c3b47084c12b19844ccfd005999
I20260925 10:58:28.107424 16127 ts_tablet_manager.cc:1939] T 827e5c3b47084c12b19844ccfd005999 P 3334a8fea2ea46bbadf2f9c28cd5e268: Deleting consensus metadata
I20260925 10:58:28.109346 15620 catalog_manager.cc:4841] TS 3334a8fea2ea46bbadf2f9c28cd5e268 (127.31.62.130:35557): tablet 827e5c3b47084c12b19844ccfd005999 (table test-table [id=97513378bf2a4b998e84e6edcc9c19db]) successfully deleted
I20260925 10:58:28.109884 16125 ts_tablet_manager.cc:1918] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:58:28.110112 16125 log.cc:1196] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/827e5c3b47084c12b19844ccfd005999
I20260925 10:58:28.110703 16125 ts_tablet_manager.cc:1939] T 827e5c3b47084c12b19844ccfd005999 P 0b1719c04ff84ab59df62b3e866578ea: Deleting consensus metadata
I20260925 10:58:28.111267 16097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:28.076909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43360 (local address 127.31.62.129:44585)
0925 10:58:28.077092 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:28.077098 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:28.092129 (+ 15031us) server_negotiation.cc:407] Connection header received
0925 10:58:28.092640 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:28.092656 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:28.092744 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:28.092978 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:28.095216 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:28.096431 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:28.098590 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:28.099340 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:28.099458 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:28.103330 (+ 3872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:28.103348 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:28.103370 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:28.103417 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:28.107330 (+ 3913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:28.108396 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:28.108407 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:28.108411 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:28.108591 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:28.109132 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:28.109137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:28.109141 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:28.109632 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:28.109944 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:28.110399 (+ 455us) server_negotiation.cc:299] Negotiation successful
0925 10:58:28.110663 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 10:58:28.111991 15815 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:58:28.112684 15617 catalog_manager.cc:4841] TS 0b1719c04ff84ab59df62b3e866578ea (127.31.62.129:44585): tablet 827e5c3b47084c12b19844ccfd005999 (table test-table [id=97513378bf2a4b998e84e6edcc9c19db]) successfully deleted
I20260925 10:58:28.215895 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 15671
I20260925 10:58:28.231040 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 15824
I20260925 10:58:28.245903 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:28.231966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42512 (local address 127.31.62.130:35557)
0925 10:58:28.232162 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:28.232168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:28.232191 (+ 23us) server_negotiation.cc:407] Connection header received
0925 10:58:28.232255 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:28.232261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:28.232328 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:28.232441 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:28.233829 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:28.234677 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:28.235986 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:28.236446 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:28.236569 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:28.239305 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:28.239321 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:28.239337 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:28.239379 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:28.242367 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:28.243273 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:28.243281 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:28.243284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:28.243464 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:28.243866 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:28.243869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:28.243872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:28.244261 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:28.244624 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:28.245243 (+ 619us) server_negotiation.cc:299] Negotiation successful
0925 10:58:28.245564 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 10:58:28.246649 15948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:58:28.360144 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 15824
I20260925 10:58:28.375734 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 15956
I20260925 10:58:28.395802 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:28.376510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44594 (local address 127.31.62.131:43497)
0925 10:58:28.376692 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 10:58:28.376697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:58:28.376716 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:58:28.376833 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:58:28.376837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:58:28.376947 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:58:28.377043 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:58:28.378404 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:28.379125 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:28.380415 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:58:28.381060 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:58:28.381159 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:58:28.389519 (+ 8360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:58:28.389538 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:58:28.389554 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:58:28.389617 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:58:28.392512 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:28.393538 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:28.393546 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:28.393549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:28.393699 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:58:28.394222 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:58:28.394226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:58:28.394228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:58:28.394623 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:58:28.394845 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0925 10:58:28.395255 (+ 410us) server_negotiation.cc:299] Negotiation successful
0925 10:58:28.395433 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 10:58:28.396482 16080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:58:28.484664 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 15956
I20260925 10:58:28.497881 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 15600
I20260925 10:58:28.499075 15664 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:58:28.611867 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 15600
I20260925 10:58:48.629730 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:34111
--webserver_interface=127.31.62.190
--webserver_port=37889
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:34111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:58:48.958021 16133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:58:48.958330 16133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:58:48.958451 16133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:58:48.966305 16133 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 10:58:48.966428 16133 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 10:58:48.966467 16133 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 10:58:48.966543 16133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:58:48.966619 16133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:58:48.966717 16133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:58:48.966784 16133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:58:48.966857 16133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:58:48.977510 16133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:34111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:34111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=37889
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:58:48.978842 16133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:58:48.980155 16133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:58:48.992473 16141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:48.992534 16139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:48.992496 16138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:58:48.993180 16133 server_base.cc:1034] running on GCE node
I20260925 10:58:48.993983 16133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:58:48.995441 16133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:58:48.996667 16133 hybrid_clock.cc:648] HybridClock initialized: now 1790333928996595 us; error 48 us; skew 500 ppm
I20260925 10:58:48.997051 16133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:58:48 dist-test-slave-nx7s krb5kdc[15047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333928, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:58:49.003614 16133 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:58:49.005354 16133 webserver.cc:460] Webserver started at http://127.31.62.190:37889/ using document root <none> and password file <none>
I20260925 10:58:49.006084 16133 fs_manager.cc:359] Metadata directory not provided
I20260925 10:58:49.006191 16133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:58:49.011934 16133 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 10:58:49.015110 16148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:49.016654 16133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:58:49.016832 16133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5"
format_stamp: "Formatted at 2026-09-25 10:58:03 on dist-test-slave-nx7s"
server_key: "c23fd8b3a87778a20fbd5777e78f3d1f"
server_key_iv: "9822d4d372faf87b1bbbd4235598c1a6"
server_key_version: "encryptionkey@0"
I20260925 10:58:49.017997 16133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:58:49.041291 16133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:58:49.045923 16133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:58:49.046197 16133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:58:49.064116 16133 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:34111
I20260925 10:58:49.064148 16199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:34111 every 8 connection(s)
I20260925 10:58:49.065775 16133 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:58:49.073163 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 16133
I20260925 10:58:49.073808 16200 sys_catalog.cc:263] Verifying existing consensus state
I20260925 10:58:49.074126 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:44585
--local_ip_for_outbound_sockets=127.31.62.129
--tserver_master_addrs=127.31.62.190:34111
--webserver_port=46689
--webserver_interface=127.31.62.129
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:58:49.077368 16200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Bootstrap starting.
I20260925 10:58:49.098133 16200 log.cc:824] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Log is configured to *not* fsync() on all Append() calls
I20260925 10:58:49.114429 16200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 10:58:49.114986 16200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Bootstrap complete.
I20260925 10:58:49.124167 16200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } }
I20260925 10:58:49.125162 16200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa5d569e58449a3ad573f2cf3d3ffe5, State: Initialized, Role: FOLLOWER
I20260925 10:58:49.125771 16200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } }
I20260925 10:58:49.126089 16200 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:58:49.126200 16200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:58:49.126361 16200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 2 FOLLOWER]: Advancing to term 3
I20260925 10:58:49.130314 16200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } }
I20260925 10:58:49.130805 16200 leader_election.cc:302] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1aa5d569e58449a3ad573f2cf3d3ffe5; no voters:
I20260925 10:58:49.131471 16200 leader_election.cc:288] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 10:58:49.131611 16204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 10:58:49.132627 16204 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [term 3 LEADER]: Becoming Leader. State: Replica: 1aa5d569e58449a3ad573f2cf3d3ffe5, State: Running, Role: LEADER
I20260925 10:58:49.133188 16204 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } }
I20260925 10:58:49.133668 16200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:58:49.135550 16205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } } }
I20260925 10:58:49.135913 16205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:58:49.137539 16209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:58:49.137373 16206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aa5d569e58449a3ad573f2cf3d3ffe5. Latest consensus state: current_term: 3 leader_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa5d569e58449a3ad573f2cf3d3ffe5" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 34111 } } }
I20260925 10:58:49.138057 16206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5 [sys.catalog]: This master's current role is: LEADER
I20260925 10:58:49.146700 16209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:58:49.148337 16209 catalog_manager.cc:1259] Loaded cluster ID: 76e51fd4b12043608d673ea1428510ee
I20260925 10:58:49.148515 16209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:58:49.152705 16209 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:58:49.154548 16209 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Loaded TSK: 0
I20260925 10:58:49.154811 16209 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1aa5d569e58449a3ad573f2cf3d3ffe5: Loaded TSK: 1
F20260925 10:58:49.156119 16209 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9072f2f9ad google::LogMessage::Fail() at ??:0
@ 0x7f9072f3288a google::LogMessage::SendToLog() at ??:0
@ 0x7f9072f2f4de google::LogMessage::Flush() at ??:0
@ 0x7f9072f30a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9076c937d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9076c8273e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f90889621c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f9088961783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f90889ed2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f90889ed0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f9088a375e9 std::function<>::operator()() at ??:0
@ 0x7f9088965cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f9088963c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f90889eb610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f90889eb439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260925 10:58:49.225243 16225 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f9075b7b345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9075b7e91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f9075b7e799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f9075b59eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9078e336db start_thread at ??:0
@ 0x7f9071dc961f clone at ??:0
W20260925 10:58:49.478363 16202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:58:49.478667 16202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:58:49.478801 16202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:58:49.486223 16202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:58:49.486332 16202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:58:49.486470 16202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:58:49.496325 16202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:44585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=46689
--tserver_master_addrs=127.31.62.190:34111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:58:49.497763 16202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:58:49.499114 16202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:58:49.512351 16248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:49.512374 16247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:49.512264 16250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:58:49.513047 16202 server_base.cc:1034] running on GCE node
I20260925 10:58:49.513653 16202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:58:49.514588 16202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:58:49.515800 16202 hybrid_clock.cc:648] HybridClock initialized: now 1790333929515750 us; error 38 us; skew 500 ppm
I20260925 10:58:49.516182 16202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:58:49 dist-test-slave-nx7s krb5kdc[15047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333929, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:58:49.519279 16202 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:58:49.520725 16202 webserver.cc:460] Webserver started at http://127.31.62.129:46689/ using document root <none> and password file <none>
I20260925 10:58:49.521435 16202 fs_manager.cc:359] Metadata directory not provided
I20260925 10:58:49.521548 16202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:58:49.526966 16202 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260925 10:58:49.529824 16257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:49.531193 16202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 10:58:49.531347 16202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "0b1719c04ff84ab59df62b3e866578ea"
format_stamp: "Formatted at 2026-09-25 10:58:03 on dist-test-slave-nx7s"
server_key: "bbf81a67656c5fa9c5534108de43e8cc"
server_key_iv: "3c1aa285889b4c2a189d39d3b3bba724"
server_key_version: "encryptionkey@0"
I20260925 10:58:49.532379 16202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:58:49.550690 16202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:58:49.554908 16202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:58:49.555156 16202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:58:49.556159 16202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:58:49.557637 16202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:58:49.557736 16202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:49.557852 16202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:58:49.557927 16202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:49.588769 16202 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:44585
I20260925 10:58:49.588884 16369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:44585 every 8 connection(s)
I20260925 10:58:49.590222 16202 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:58:49.595135 16372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:49.593203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:52837)
0925 10:58:49.594006 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:49.594232 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":273,"threads_started":1}
W20260925 10:58:49.595854 16370 heartbeater.cc:644] Failed to heartbeat to 127.31.62.190:34111 (0 consecutive failures): Network error: Failed to ping master at 127.31.62.190:34111: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
I20260925 10:58:49.596730 16372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:49.596407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:51955)
0925 10:58:49.596544 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:49.596622 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 10:58:49.597657 16372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:49.597304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:50767)
0925 10:58:49.597466 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:49.597543 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260925 10:58:49.597892 16370 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 10:58:49.598315 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 16202
I20260925 10:58:49.598968 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:35557
--local_ip_for_outbound_sockets=127.31.62.130
--tserver_master_addrs=127.31.62.190:34111
--webserver_port=45021
--webserver_interface=127.31.62.130
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:58:49.921190 16373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:58:49.921525 16373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:58:49.921636 16373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:58:49.928982 16373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:58:49.929112 16373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:58:49.929276 16373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:58:49.939500 16373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:35557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=45021
--tserver_master_addrs=127.31.62.190:34111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:58:49.940920 16373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:58:49.942267 16373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:58:49.955538 16379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:49.955505 16378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:49.955587 16381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:58:49.956180 16373 server_base.cc:1034] running on GCE node
I20260925 10:58:49.956770 16373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:58:49.957803 16373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:58:49.959039 16373 hybrid_clock.cc:648] HybridClock initialized: now 1790333929958986 us; error 42 us; skew 500 ppm
I20260925 10:58:49.959434 16373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:58:49 dist-test-slave-nx7s krb5kdc[15047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333929, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:58:49.962666 16373 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:58:49.964242 16373 webserver.cc:460] Webserver started at http://127.31.62.130:45021/ using document root <none> and password file <none>
I20260925 10:58:49.965029 16373 fs_manager.cc:359] Metadata directory not provided
I20260925 10:58:49.965145 16373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:58:49.971276 16373 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 10:58:49.974375 16388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:49.975929 16373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:58:49.976094 16373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "3334a8fea2ea46bbadf2f9c28cd5e268"
format_stamp: "Formatted at 2026-09-25 10:58:04 on dist-test-slave-nx7s"
server_key: "1ab346a62880dec4b7f75f55e816c0fd"
server_key_iv: "62e19caf289c28aae032315ea65244f1"
server_key_version: "encryptionkey@0"
I20260925 10:58:49.977113 16373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:58:49.996255 16373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:58:50.000571 16373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:58:50.000837 16373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:58:50.001892 16373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:58:50.003305 16373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:58:50.003403 16373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:50.003496 16373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:58:50.003566 16373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:50.033224 16373 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:35557
I20260925 10:58:50.033278 16500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:35557 every 8 connection(s)
I20260925 10:58:50.034657 16373 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:58:50.039682 16503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:50.037720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:53705)
0925 10:58:50.038453 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:50.038677 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":284,"threads_started":1}
W20260925 10:58:50.040437 16501 heartbeater.cc:644] Failed to heartbeat to 127.31.62.190:34111 (0 consecutive failures): Network error: Failed to ping master at 127.31.62.190:34111: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
I20260925 10:58:50.041270 16503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:50.040931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:59365)
0925 10:58:50.041072 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:50.041147 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260925 10:58:50.041493 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 16373
I20260925 10:58:50.042029 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:43497
--local_ip_for_outbound_sockets=127.31.62.131
--tserver_master_addrs=127.31.62.190:34111
--webserver_port=43259
--webserver_interface=127.31.62.131
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 10:58:50.042157 16503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:50.041789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:56699)
0925 10:58:50.041921 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:50.042041 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260925 10:58:50.042372 16501 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 10:58:50.364125 16504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:58:50.364441 16504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:58:50.364573 16504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:58:50.372167 16504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:58:50.372299 16504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:58:50.372428 16504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:58:50.382720 16504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:43497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=43259
--tserver_master_addrs=127.31.62.190:34111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:58:50.383996 16504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:58:50.385336 16504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:58:50.398977 16509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:50.399024 16510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:58:50.398972 16512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:58:50.400030 16504 server_base.cc:1034] running on GCE node
I20260925 10:58:50.400647 16504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:58:50.401647 16504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:58:50.402863 16504 hybrid_clock.cc:648] HybridClock initialized: now 1790333930402804 us; error 50 us; skew 500 ppm
I20260925 10:58:50.403234 16504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:58:50 dist-test-slave-nx7s krb5kdc[15047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333930, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:58:50.406445 16504 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:58:50.407992 16504 webserver.cc:460] Webserver started at http://127.31.62.131:43259/ using document root <none> and password file <none>
I20260925 10:58:50.408679 16504 fs_manager.cc:359] Metadata directory not provided
I20260925 10:58:50.408798 16504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:58:50.414397 16504 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260925 10:58:50.417158 16519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:50.418551 16504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 10:58:50.418715 16504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "32389eab881f4599b46dba9d30079bc1"
format_stamp: "Formatted at 2026-09-25 10:58:04 on dist-test-slave-nx7s"
server_key: "dfa42167be4081a996b9041a7ce50ac1"
server_key_iv: "bebe7e8b8c7e5ff28f9714f3fcf26bb8"
server_key_version: "encryptionkey@0"
I20260925 10:58:50.419724 16504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:58:50.445374 16504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:58:50.449895 16504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:58:50.450167 16504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:58:50.451170 16504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:58:50.452556 16504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:58:50.452651 16504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:50.452756 16504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:58:50.452824 16504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:58:50.483728 16504 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:43497
I20260925 10:58:50.483794 16631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:43497 every 8 connection(s)
I20260925 10:58:50.485241 16504 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:58:50.490105 16634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:50.488190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:57695)
0925 10:58:50.488952 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:50.489204 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":284,"threads_started":1}
W20260925 10:58:50.490792 16632 heartbeater.cc:644] Failed to heartbeat to 127.31.62.190:34111 (0 consecutive failures): Network error: Failed to ping master at 127.31.62.190:34111: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
I20260925 10:58:50.491663 16634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:50.491359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:37625)
0925 10:58:50.491485 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:50.491554 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 10:58:50.492409 16634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:50.492133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:35957)
0925 10:58:50.492242 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:50.492310 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260925 10:58:50.492643 16632 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 10:58:50.494606 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 16504
I20260925 10:58:50.599222 16636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:50.598403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:44643)
0925 10:58:50.598885 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:50.598982 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":232,"threads_started":1}
I20260925 10:58:51.043807 16637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:51.043010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:52369)
0925 10:58:51.043466 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:51.043574 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":209,"threads_started":1}
I20260925 10:58:51.494074 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:51.493238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:56587)
0925 10:58:51.493733 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:51.493832 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":248,"threads_started":1}
I20260925 10:58:51.601274 16639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:51.600368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:57119)
0925 10:58:51.600900 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:51.601079 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":230,"threads_started":1}
I20260925 10:58:52.045538 16640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:52.044787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:60097)
0925 10:58:52.045262 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:52.045371 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":243,"threads_started":1}
I20260925 10:58:52.495599 16641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:52.494932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:52541)
0925 10:58:52.495348 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:52.495428 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":199,"threads_started":1}
I20260925 10:58:52.603096 16642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:52.602360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:35869)
0925 10:58:52.602795 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:52.602883 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":215,"threads_started":1}
I20260925 10:58:53.047163 16643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:53.046347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:55443)
0925 10:58:53.046896 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:53.046993 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":306,"threads_started":1}
I20260925 10:58:53.501677 16644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:53.496568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:53225)
0925 10:58:53.501317 (+ 4749us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:53.501434 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4614,"thread_start_us":4486,"threads_started":1}
I20260925 10:58:53.604948 16645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:53.604181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:37993)
0925 10:58:53.604639 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:53.604726 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":229,"threads_started":1}
I20260925 10:58:54.049108 16646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:54.048126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:42981)
0925 10:58:54.048604 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:54.048743 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":232,"threads_started":1}
I20260925 10:58:54.503382 16647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:54.502645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:45019)
0925 10:58:54.503128 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:54.503211 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":214,"threads_started":1}
I20260925 10:58:54.606590 16648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:54.605845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:38257)
0925 10:58:54.606302 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:54.606387 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":227,"threads_started":1}
I20260925 10:58:55.050999 16649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:55.050179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:35289)
0925 10:58:55.050687 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:55.050777 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":259,"threads_started":1}
I20260925 10:58:55.505213 16650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:55.504399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:58485)
0925 10:58:55.504939 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:55.505034 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":250,"threads_started":1}
I20260925 10:58:55.608278 16651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:55.607527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:51861)
0925 10:58:55.608017 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:55.608114 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":276,"threads_started":1}
I20260925 10:58:56.052625 16652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:56.051935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:39529)
0925 10:58:56.052372 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:56.052463 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":221,"threads_started":1}
I20260925 10:58:56.506975 16653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:56.506153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:46297)
0925 10:58:56.506636 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:56.506735 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":250,"threads_started":1}
I20260925 10:58:56.610091 16654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:56.609249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:39855)
0925 10:58:56.609761 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:56.609856 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":215,"threads_started":1}
I20260925 10:58:57.054383 16655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:57.053598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:41193)
0925 10:58:57.054086 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:57.054174 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":237,"threads_started":1}
I20260925 10:58:57.508906 16656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:57.508031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:53917)
0925 10:58:57.508551 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:57.508648 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":255,"threads_started":1}
I20260925 10:58:57.611713 16657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:57.610998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:57451)
0925 10:58:57.611460 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:57.611546 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":223,"threads_started":1}
I20260925 10:58:58.056172 16658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:58.055392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:36947)
0925 10:58:58.055860 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:58.055990 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":241,"threads_started":1}
I20260925 10:58:58.510732 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:58.509977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:47701)
0925 10:58:58.510477 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:58.510574 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":248,"threads_started":1}
I20260925 10:58:58.613420 16660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:58.612652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:59045)
0925 10:58:58.613160 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:58.613247 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":254,"threads_started":1}
I20260925 10:58:59.057915 16661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:59.057117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:57873)
0925 10:58:59.057637 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:59.057731 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":251,"threads_started":1}
I20260925 10:58:59.512462 16662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:59.511692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:45975)
0925 10:58:59.512166 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:59.512264 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":234,"threads_started":1}
I20260925 10:58:59.615130 16663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:58:59.614352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:48797)
0925 10:58:59.614819 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0925 10:58:59.614966 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":260,"threads_started":1}
I20260925 10:59:00.059646 16664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:00.058901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:51207)
0925 10:59:00.059363 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:00.059465 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":244,"threads_started":1}
I20260925 10:59:00.514088 16665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:00.513318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:39395)
0925 10:59:00.513809 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:00.513905 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":237,"threads_started":1}
I20260925 10:59:00.616816 16666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:00.616040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:40037)
0925 10:59:00.616540 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:00.616637 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":217,"threads_started":1}
I20260925 10:59:01.061455 16667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:01.060685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:58339)
0925 10:59:01.061173 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:01.061270 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":276,"threads_started":1}
I20260925 10:59:01.515803 16668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:01.515019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:37203)
0925 10:59:01.515525 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:01.515633 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":237,"threads_started":1}
I20260925 10:59:01.618448 16669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:01.617726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:58109)
0925 10:59:01.618198 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:01.618288 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":238,"threads_started":1}
I20260925 10:59:02.063071 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:02.062348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:43953)
0925 10:59:02.062784 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:02.062863 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":224,"threads_started":1}
I20260925 10:59:02.517446 16671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:02.516723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:58811)
0925 10:59:02.517194 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:02.517281 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":255,"threads_started":1}
I20260925 10:59:02.620201 16672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:02.619356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:42629)
0925 10:59:02.619904 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:02.620009 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":258,"threads_started":1}
I20260925 10:59:03.064996 16673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:03.064074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:53237)
0925 10:59:03.064625 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:03.064726 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":258,"threads_started":1}
I20260925 10:59:03.519263 16674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:03.518382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:42673)
0925 10:59:03.518908 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:03.519020 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":238,"threads_started":1}
I20260925 10:59:03.621856 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:03.621111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:59995)
0925 10:59:03.621577 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:03.621667 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":232,"threads_started":1}
I20260925 10:59:04.066838 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:04.066052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:40469)
0925 10:59:04.066559 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:04.066656 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":272,"threads_started":1}
I20260925 10:59:04.521059 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:04.520216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:33349)
0925 10:59:04.520718 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:04.520827 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":244,"threads_started":1}
I20260925 10:59:04.623731 16678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:04.622926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.129:57671)
0925 10:59:04.623420 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:04.623511 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":285,"threads_started":1}
I20260925 10:59:05.068723 16679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:05.067986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.130:40753)
0925 10:59:05.068447 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:05.068542 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":222,"threads_started":1}
I20260925 10:59:05.504550 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 16202
I20260925 10:59:05.522893 16681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:05.522147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:34111 (local address 127.31.62.131:59187)
0925 10:59:05.522599 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:05.522711 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.190:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":237,"threads_started":1}
I20260925 10:59:05.523389 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:05.505758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56158 (local address 127.31.62.129:44585)
0925 10:59:05.506334 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:05.506348 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:05.506397 (+ 49us) server_negotiation.cc:407] Connection header received
0925 10:59:05.508032 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:05.508045 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:05.508505 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:05.508818 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:05.510282 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:05.511670 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:05.513110 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:05.513844 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:05.514017 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:05.516839 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:05.516929 (+ 90us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:05.516949 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:05.516993 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:05.519881 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:05.520604 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:05.520611 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:05.520618 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:05.520792 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:05.521297 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:05.521301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:05.521303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:05.521859 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:05.522192 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:05.522647 (+ 455us) server_negotiation.cc:299] Negotiation successful
0925 10:59:05.523121 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":259,"threads_started":1}
I20260925 10:59:05.524761 16365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:05.609084 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 16202
I20260925 10:59:05.622851 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 16373
I20260925 10:59:05.641242 16683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:05.623798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55812 (local address 127.31.62.130:35557)
0925 10:59:05.624315 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:05.624330 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:05.624369 (+ 39us) server_negotiation.cc:407] Connection header received
0925 10:59:05.625949 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:05.625961 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:05.626389 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:05.626692 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:05.628077 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:05.629427 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:05.630680 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:05.631368 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:05.631511 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:05.634567 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:05.634575 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:05.634593 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:05.634638 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:05.637594 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:05.638411 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:05.638418 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:05.638426 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:05.638608 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:05.639074 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:05.639079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:05.639081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:05.639693 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:05.640035 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:05.640419 (+ 384us) server_negotiation.cc:299] Negotiation successful
0925 10:59:05.640783 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":251,"threads_started":1}
I20260925 10:59:05.642705 16496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:05.725324 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 16373
I20260925 10:59:05.739701 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 16504
I20260925 10:59:05.756939 16685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:05.740696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34416 (local address 127.31.62.131:43497)
0925 10:59:05.741277 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:05.741291 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:05.741334 (+ 43us) server_negotiation.cc:407] Connection header received
0925 10:59:05.742942 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:05.742951 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:05.743380 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:05.743695 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:05.745000 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:05.746272 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:05.747518 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:05.748172 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:05.748332 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:05.750888 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:05.750897 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:05.750912 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:05.750954 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:05.753841 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:05.754473 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:05.754480 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:05.754486 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:05.754650 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:05.754989 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:05.754994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:05.754996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:05.755576 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:05.755830 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:05.756154 (+ 324us) server_negotiation.cc:299] Negotiation successful
0925 10:59:05.756542 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":285,"threads_started":1}
I20260925 10:59:05.758416 16627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:05.841826 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 16504
2026-09-25T10:59:05Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64412 ms)
[----------] 30 tests from SecurityITest (298418 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:05 dist-test-slave-nx7s krb5kdc[16689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:05 dist-test-slave-nx7s krb5kdc[16689](info): set up 2 sockets
Sep 25 10:59:05 dist-test-slave-nx7s krb5kdc[16689](info): commencing operation
krb5kdc: starting...
W20260925 10:59:06.865191 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:07 dist-test-slave-nx7s krb5kdc[16689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:59:07.104748 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:42381
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:42381
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:07.442631 16702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:07.442951 16702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:07.443073 16702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:07.450564 16702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:07.450691 16702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:07.450738 16702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:07.450826 16702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:07.450884 16702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:07.461107 16702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:42381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:42381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:07.462636 16702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:07.464082 16702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:07.475801 16708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:07.475868 16710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:07.476125 16702 server_base.cc:1034] running on GCE node
W20260925 10:59:07.475831 16707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:07.477264 16702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:07.478775 16702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:07.479995 16702 hybrid_clock.cc:648] HybridClock initialized: now 1790333947479938 us; error 43 us; skew 500 ppm
I20260925 10:59:07.480336 16702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:07 dist-test-slave-nx7s krb5kdc[16689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333947, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:07.483632 16702 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:59:07.485410 16702 webserver.cc:460] Webserver started at http://127.31.62.190:34615/ using document root <none> and password file <none>
I20260925 10:59:07.486140 16702 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:07.486284 16702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:07.486601 16702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:07.489919 16702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "6ae84726585e4c41809cd620b8cc1f61"
format_stamp: "Formatted at 2026-09-25 10:59:07 on dist-test-slave-nx7s"
server_key: "ea826cf2fbe1893c5a6bf3b1ab9d9707"
server_key_iv: "de71c94fddbff5a3979ff286cd41e7aa"
server_key_version: "encryptionkey@0"
I20260925 10:59:07.490725 16702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "6ae84726585e4c41809cd620b8cc1f61"
format_stamp: "Formatted at 2026-09-25 10:59:07 on dist-test-slave-nx7s"
server_key: "ea826cf2fbe1893c5a6bf3b1ab9d9707"
server_key_iv: "de71c94fddbff5a3979ff286cd41e7aa"
server_key_version: "encryptionkey@0"
I20260925 10:59:07.496730 16702 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260925 10:59:07.500765 16717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:07.502321 16702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:59:07.502497 16702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "6ae84726585e4c41809cd620b8cc1f61"
format_stamp: "Formatted at 2026-09-25 10:59:07 on dist-test-slave-nx7s"
server_key: "ea826cf2fbe1893c5a6bf3b1ab9d9707"
server_key_iv: "de71c94fddbff5a3979ff286cd41e7aa"
server_key_version: "encryptionkey@0"
I20260925 10:59:07.502698 16702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:07.538463 16702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:07.543246 16702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:07.543537 16702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:07.561854 16702 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:42381
I20260925 10:59:07.561885 16768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:42381 every 8 connection(s)
I20260925 10:59:07.563273 16702 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:07.567250 16769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:07.569813 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 16702
I20260925 10:59:07.570060 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:07.570827 31994 external_mini_cluster.cc:1428] Setting key c0a846d8d1cba3167041d99b81b7bd2d
I20260925 10:59:07.579960 16769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61: Bootstrap starting.
I20260925 10:59:07.583824 16769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:07.585438 16769 log.cc:824] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61: Log is configured to *not* fsync() on all Append() calls
Sep 25 10:59:07 dist-test-slave-nx7s krb5kdc[16689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:59:07.588701 16769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61: No bootstrap required, opened a new log
I20260925 10:59:07.595431 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:07.573065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49156 (local address 127.31.62.190:42381)
0925 10:59:07.573844 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:07.573857 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:07.573901 (+ 44us) server_negotiation.cc:407] Connection header received
0925 10:59:07.575625 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:07.575649 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:07.576078 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:07.576435 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:07.578248 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:07.579653 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:07.581628 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:07.582342 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:07.582513 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:07.587877 (+ 5364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:07.587885 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:07.587902 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:07.587939 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:07.591188 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:07.591925 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:07.591931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:07.591941 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:07.592110 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:07.592499 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:07.592502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:07.592504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:07.593148 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:07.593439 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:07.593920 (+ 481us) server_negotiation.cc:299] Negotiation successful
0925 10:59:07.594339 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":356,"threads_started":1}
I20260925 10:59:07.597112 16769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae84726585e4c41809cd620b8cc1f61" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42381 } }
I20260925 10:59:07.597505 16769 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:07.597602 16769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae84726585e4c41809cd620b8cc1f61, State: Initialized, Role: FOLLOWER
I20260925 10:59:07.598193 16769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae84726585e4c41809cd620b8cc1f61" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42381 } }
I20260925 10:59:07.598469 16769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:07.598655 16769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:07.598806 16769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:07.600970 16769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae84726585e4c41809cd620b8cc1f61" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42381 } }
I20260925 10:59:07.601437 16769 leader_election.cc:302] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ae84726585e4c41809cd620b8cc1f61; no voters:
I20260925 10:59:07.602051 16769 leader_election.cc:288] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:07.602200 16774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:07.602816 16774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [term 1 LEADER]: Becoming Leader. State: Replica: 6ae84726585e4c41809cd620b8cc1f61, State: Running, Role: LEADER
I20260925 10:59:07.603499 16774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae84726585e4c41809cd620b8cc1f61" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42381 } }
I20260925 10:59:07.605185 16769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:07.606483 16776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ae84726585e4c41809cd620b8cc1f61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae84726585e4c41809cd620b8cc1f61" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42381 } } }
I20260925 10:59:07.606906 16776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:07.606527 16775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ae84726585e4c41809cd620b8cc1f61. Latest consensus state: current_term: 1 leader_uuid: "6ae84726585e4c41809cd620b8cc1f61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae84726585e4c41809cd620b8cc1f61" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 42381 } } }
I20260925 10:59:07.607570 16775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:07.608383 16780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:07.614248 16780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:07.623833 16780 catalog_manager.cc:1347] Generated new cluster ID: f51e7dc5ce234c6cb6ba201b76930adf
I20260925 10:59:07.623986 16780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:07.649557 16780 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:07.650990 16780 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:07.663801 16780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61: Generated new TSK 0
I20260925 10:59:07.664736 16780 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:59:07.729357 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42381
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:08.057744 16795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:08.058040 16795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:08.058157 16795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:08.065507 16795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:08.065634 16795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:08.065775 16795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:59:08.075856 16795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42381
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:08.077219 16795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:08.078563 16795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:08.091372 16803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:08.091339 16801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:08.091445 16800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:08.097667 16795 server_base.cc:1034] running on GCE node
I20260925 10:59:08.098301 16795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:08.099277 16795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:08.100508 16795 hybrid_clock.cc:648] HybridClock initialized: now 1790333948100438 us; error 58 us; skew 500 ppm
I20260925 10:59:08.100957 16795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:08 dist-test-slave-nx7s krb5kdc[16689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333948, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:08.104566 16795 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:59:08.106357 16795 webserver.cc:460] Webserver started at http://127.31.62.129:34463/ using document root <none> and password file <none>
I20260925 10:59:08.107185 16795 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:08.107307 16795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:08.107654 16795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:08.111259 16795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "a8e21d6a5f834c749472bf82449cf911"
format_stamp: "Formatted at 2026-09-25 10:59:08 on dist-test-slave-nx7s"
server_key: "8f40ef19ae8afaa60044cb2942eda753"
server_key_iv: "883a4d9883b38c16e5a6d6455138a387"
server_key_version: "encryptionkey@0"
I20260925 10:59:08.112171 16795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "a8e21d6a5f834c749472bf82449cf911"
format_stamp: "Formatted at 2026-09-25 10:59:08 on dist-test-slave-nx7s"
server_key: "8f40ef19ae8afaa60044cb2942eda753"
server_key_iv: "883a4d9883b38c16e5a6d6455138a387"
server_key_version: "encryptionkey@0"
I20260925 10:59:08.117877 16795 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 10:59:08.122097 16810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:08.123785 16795 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260925 10:59:08.124039 16795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "a8e21d6a5f834c749472bf82449cf911"
format_stamp: "Formatted at 2026-09-25 10:59:08 on dist-test-slave-nx7s"
server_key: "8f40ef19ae8afaa60044cb2942eda753"
server_key_iv: "883a4d9883b38c16e5a6d6455138a387"
server_key_version: "encryptionkey@0"
I20260925 10:59:08.124267 16795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:08.157354 16795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:08.161532 16795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:08.161839 16795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:08.162863 16795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:08.164273 16795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:08.164350 16795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:08.164453 16795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:08.164511 16795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:08.195264 16795 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:40937
I20260925 10:59:08.195313 16922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:40937 every 8 connection(s)
I20260925 10:59:08.196606 16795 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:59:08.202901 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 16795
I20260925 10:59:08.203136 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:59:08.203752 31994 external_mini_cluster.cc:1428] Setting key a56ac53384a0d08c2a6ee10368c78d79
Sep 25 10:59:08 dist-test-slave-nx7s krb5kdc[16689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333948, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:59:08.218561 16925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:08.199450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:60161 (local address 127.31.62.190:42381)
0925 10:59:08.199956 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:08.199960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:08.200813 (+ 853us) server_negotiation.cc:407] Connection header received
0925 10:59:08.203058 (+ 2245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:08.203064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:08.203153 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:08.203255 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:08.205093 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:08.206183 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:08.207886 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:08.208294 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:08.208419 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:08.212229 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:08.212236 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:08.212241 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:08.212285 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:08.215091 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:08.216054 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:08.216059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:08.216061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:08.216198 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:08.216681 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:08.216685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:08.216687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:08.217029 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:08.217144 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:08.218073 (+ 929us) server_negotiation.cc:299] Negotiation successful
0925 10:59:08.218259 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":257,"threads_started":1}
I20260925 10:59:08.219934 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:08.199935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:42381 (local address 127.31.62.129:60161)
0925 10:59:08.200635 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:08.200679 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:08.202748 (+ 2069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:08.203530 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:08.203540 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:08.204128 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:08.204902 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:08.204932 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:08.206332 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:08.206338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:08.207744 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:08.207762 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:08.208533 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:08.208542 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:08.208802 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:08.209513 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:59:08.209546 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:08.211860 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:08.215261 (+ 3401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:08.215271 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:08.215293 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:08.215909 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:08.216300 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:08.216304 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:08.216308 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:08.216554 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:08.217138 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:08.217147 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:08.217620 (+ 473us) client_negotiation.cc:769] Sending connection context
0925 10:59:08.217942 (+ 322us) client_negotiation.cc:240] Negotiation successful
0925 10:59:08.218515 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":278,"threads_started":1}
I20260925 10:59:08.221627 16923 heartbeater.cc:347] Connected to a master server at 127.31.62.190:42381
I20260925 10:59:08.221976 16923 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:08.222877 16923 heartbeater.cc:511] Master 127.31.62.190:42381 requested a full tablet report, sending...
I20260925 10:59:08.225761 16734 ts_manager.cc:194] Registered new tserver with Master: a8e21d6a5f834c749472bf82449cf911 (127.31.62.129:40937)
I20260925 10:59:08.228200 16734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:60161
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:59:08.261377 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42381
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:08.588603 16929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:08.588943 16929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:08.589067 16929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:08.596426 16929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:08.596541 16929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:08.596693 16929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:59:08.606855 16929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42381
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:08.608305 16929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:08.609800 16929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:08.623930 16937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:08.624023 16934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:08.624004 16935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:08.624629 16929 server_base.cc:1034] running on GCE node
I20260925 10:59:08.625276 16929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:08.626200 16929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:08.627401 16929 hybrid_clock.cc:648] HybridClock initialized: now 1790333948627331 us; error 53 us; skew 500 ppm
I20260925 10:59:08.627696 16929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:08 dist-test-slave-nx7s krb5kdc[16689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333948, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:08.636193 16929 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:59:08.637902 16929 webserver.cc:460] Webserver started at http://127.31.62.130:36711/ using document root <none> and password file <none>
I20260925 10:59:08.638603 16929 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:08.638708 16929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:08.638975 16929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:08.642063 16929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c"
format_stamp: "Formatted at 2026-09-25 10:59:08 on dist-test-slave-nx7s"
server_key: "0544436f9dbea9b927f1a0e91b825da1"
server_key_iv: "64884b9b0caed2438fdb9dcd15ea7b27"
server_key_version: "encryptionkey@0"
I20260925 10:59:08.642817 16929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c"
format_stamp: "Formatted at 2026-09-25 10:59:08 on dist-test-slave-nx7s"
server_key: "0544436f9dbea9b927f1a0e91b825da1"
server_key_iv: "64884b9b0caed2438fdb9dcd15ea7b27"
server_key_version: "encryptionkey@0"
I20260925 10:59:08.648216 16929 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 10:59:08.651760 16944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:08.653456 16929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:59:08.653605 16929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c"
format_stamp: "Formatted at 2026-09-25 10:59:08 on dist-test-slave-nx7s"
server_key: "0544436f9dbea9b927f1a0e91b825da1"
server_key_iv: "64884b9b0caed2438fdb9dcd15ea7b27"
server_key_version: "encryptionkey@0"
I20260925 10:59:08.653785 16929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:08.679210 16929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:08.683866 16929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:08.684125 16929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:08.685163 16929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:08.686551 16929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:08.686621 16929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:08.686735 16929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:08.686800 16929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:08.717216 16929 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:35399
I20260925 10:59:08.717283 17056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:35399 every 8 connection(s)
I20260925 10:59:08.718456 16929 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:59:08.725333 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 16929
I20260925 10:59:08.725492 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:59:08.725906 31994 external_mini_cluster.cc:1428] Setting key 2f6e6945b79483930ddb8ac331a8778b
Sep 25 10:59:08 dist-test-slave-nx7s krb5kdc[16689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333948, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:59:08.739887 17059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:08.721020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:43857 (local address 127.31.62.190:42381)
0925 10:59:08.721448 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:08.721453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:08.722509 (+ 1056us) server_negotiation.cc:407] Connection header received
0925 10:59:08.724491 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:08.724497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:08.724605 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:08.724710 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:08.726502 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:08.727386 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:08.729214 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:08.729710 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:08.729840 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:08.733837 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:08.733843 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:08.733846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:08.733885 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:08.736639 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:08.737487 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:08.737492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:08.737494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:08.737630 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:08.738073 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:08.738077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:08.738079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:08.738366 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:08.738520 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:08.739256 (+ 736us) server_negotiation.cc:299] Negotiation successful
0925 10:59:08.739510 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":202,"threads_started":1}
I20260925 10:59:08.740927 17060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:08.721560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:42381 (local address 127.31.62.130:43857)
0925 10:59:08.722327 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:08.722368 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:08.724190 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:08.725031 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:08.725040 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:08.725663 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:08.726317 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:08.726338 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:08.727542 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:08.727546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:08.729053 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:08.729067 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:08.729949 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:08.729957 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:08.730195 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:08.730976 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:59:08.731003 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:08.733548 (+ 2545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:08.736807 (+ 3259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:08.736816 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:08.736831 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:08.737340 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:08.737739 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:08.737744 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:08.737748 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:08.737951 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:08.738502 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:08.738510 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:08.738869 (+ 359us) client_negotiation.cc:769] Sending connection context
0925 10:59:08.739119 (+ 250us) client_negotiation.cc:240] Negotiation successful
0925 10:59:08.739695 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":318,"threads_started":1}
I20260925 10:59:08.742367 17057 heartbeater.cc:347] Connected to a master server at 127.31.62.190:42381
I20260925 10:59:08.742666 17057 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:08.743374 17057 heartbeater.cc:511] Master 127.31.62.190:42381 requested a full tablet report, sending...
I20260925 10:59:08.744836 16734 ts_manager.cc:194] Registered new tserver with Master: 3f2eacb05a31414c8a7f08b8fcc01b8c (127.31.62.130:35399)
I20260925 10:59:08.746012 16734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:43857
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:59:08.776147 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42381
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:34195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:09.099649 17063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:09.099999 17063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:09.100131 17063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:09.107766 17063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:09.107893 17063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:09.108045 17063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:59:09.117918 17063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:34195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:42381
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:09.119223 17063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:09.120680 17063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:09.134425 17068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:09.134603 17069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:09.135015 17071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:09.135485 17063 server_base.cc:1034] running on GCE node
I20260925 10:59:09.136086 17063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:09.137032 17063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:09.138242 17063 hybrid_clock.cc:648] HybridClock initialized: now 1790333949138195 us; error 31 us; skew 500 ppm
I20260925 10:59:09.138656 17063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:09 dist-test-slave-nx7s krb5kdc[16689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333949, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:09.142280 17063 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:59:09.143743 17063 webserver.cc:460] Webserver started at http://127.31.62.131:42569/ using document root <none> and password file <none>
I20260925 10:59:09.144416 17063 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:09.144537 17063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:09.144812 17063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:09.147683 17063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "163d31b82ce64843b09a4e237929b828"
format_stamp: "Formatted at 2026-09-25 10:59:09 on dist-test-slave-nx7s"
server_key: "d6f05eb0f06a03e3ff1b47dcb8180c66"
server_key_iv: "5dd5dd3eae9fc71da0b172972e47ad06"
server_key_version: "encryptionkey@0"
I20260925 10:59:09.148434 17063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "163d31b82ce64843b09a4e237929b828"
format_stamp: "Formatted at 2026-09-25 10:59:09 on dist-test-slave-nx7s"
server_key: "d6f05eb0f06a03e3ff1b47dcb8180c66"
server_key_iv: "5dd5dd3eae9fc71da0b172972e47ad06"
server_key_version: "encryptionkey@0"
I20260925 10:59:09.153573 17063 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260925 10:59:09.157117 17078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:09.158578 17063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 10:59:09.158739 17063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "163d31b82ce64843b09a4e237929b828"
format_stamp: "Formatted at 2026-09-25 10:59:09 on dist-test-slave-nx7s"
server_key: "d6f05eb0f06a03e3ff1b47dcb8180c66"
server_key_iv: "5dd5dd3eae9fc71da0b172972e47ad06"
server_key_version: "encryptionkey@0"
I20260925 10:59:09.158927 17063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:09.183511 17063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:09.187911 17063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:09.188190 17063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:09.189317 17063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:09.190714 17063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:09.190793 17063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:09.190912 17063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:09.190963 17063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:09.221876 17063 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:35471
I20260925 10:59:09.221941 17190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:35471 every 8 connection(s)
I20260925 10:59:09.223136 17063 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:59:09.229236 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 17063
I20260925 10:59:09.229427 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:59:09.229856 31994 external_mini_cluster.cc:1428] Setting key fcda749ada4029c9d5316df69232264c
I20260925 10:59:09.232070 16923 heartbeater.cc:503] Master 127.31.62.190:42381 was elected leader, sending a full tablet report...
Sep 25 10:59:09 dist-test-slave-nx7s krb5kdc[16689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333949, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:59:09.243592 17059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.225715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:57401 (local address 127.31.62.190:42381)
0925 10:59:09.225933 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:09.225937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:09.227105 (+ 1168us) server_negotiation.cc:407] Connection header received
0925 10:59:09.229036 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:09.229042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:09.229131 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:09.229217 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:09.230999 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.231919 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.233985 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.234365 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.234482 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.238006 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:09.238010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:09.238014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:09.238047 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:09.240656 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:09.241493 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:09.241497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:09.241499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:09.241626 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:09.242053 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:09.242057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:09.242059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:09.242351 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:09.242479 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:09.243147 (+ 668us) server_negotiation.cc:299] Negotiation successful
0925 10:59:09.243329 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 10:59:09.244874 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.226174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:42381 (local address 127.31.62.131:57401)
0925 10:59:09.226926 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:09.226966 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:09.228721 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:09.229479 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:09.229488 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:09.230084 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:09.230814 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:09.230836 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.232057 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.232061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:09.233846 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:09.233858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.234585 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.234592 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:09.234820 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.235434 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:59:09.235492 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:09.237774 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:09.240815 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:09.240823 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:09.240838 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:09.241372 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:09.241742 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:09.241746 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:09.241748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:09.241948 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:09.242458 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:09.242464 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:09.242793 (+ 329us) client_negotiation.cc:769] Sending connection context
0925 10:59:09.243043 (+ 250us) client_negotiation.cc:240] Negotiation successful
0925 10:59:09.243496 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":270,"threads_started":1}
I20260925 10:59:09.246384 17191 heartbeater.cc:347] Connected to a master server at 127.31.62.190:42381
I20260925 10:59:09.246706 17191 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:09.247412 17191 heartbeater.cc:511] Master 127.31.62.190:42381 requested a full tablet report, sending...
I20260925 10:59:09.249035 16733 ts_manager.cc:194] Registered new tserver with Master: 163d31b82ce64843b09a4e237929b828 (127.31.62.131:35471)
I20260925 10:59:09.249986 16733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:57401
I20260925 10:59:09.258504 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:59:09.275453 17059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.263013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49168 (local address 127.31.62.190:42381)
0925 10:59:09.263211 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:09.263218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:09.263627 (+ 409us) server_negotiation.cc:407] Connection header received
0925 10:59:09.263806 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:09.263811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:09.263912 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:09.263997 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:09.265480 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.266224 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.267595 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.268027 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.268138 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.270500 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:09.270505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:09.270508 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:09.270544 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:09.272935 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:09.273610 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:09.273615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:09.273617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:09.273784 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:09.274138 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:09.274141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:09.274143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:09.274474 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:09.274702 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:09.275000 (+ 298us) server_negotiation.cc:299] Negotiation successful
0925 10:59:09.275175 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 10:59:09.279712 16733 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:59:09.286654 16733 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:59:09.324331 17203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.299486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46992 (local address 127.31.62.130:35399)
0925 10:59:09.314863 (+ 15377us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:09.314874 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:09.314918 (+ 44us) server_negotiation.cc:407] Connection header received
0925 10:59:09.315047 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:09.315056 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:09.315378 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:09.315609 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:09.316941 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.318649 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.321613 (+ 2964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.323128 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.323265 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.323475 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:09.323812 (+ 337us) server_negotiation.cc:299] Negotiation successful
0925 10:59:09.323994 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15176,"thread_start_us":17123,"threads_started":1}
I20260925 10:59:09.324366 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.299666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:35399 (local address 127.0.0.1:46992)
0925 10:59:09.312956 (+ 13290us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:09.313008 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:09.313184 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:09.315937 (+ 2753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:09.315945 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:09.315981 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:09.316448 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:09.316464 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.318811 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.318816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:09.321412 (+ 2596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:09.321428 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.323330 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.323337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:09.323577 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.323633 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 10:59:09.323771 (+ 138us) client_negotiation.cc:240] Negotiation successful
0925 10:59:09.323973 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12990,"mutex_wait_us":5,"thread_start_us":17256,"threads_started":1}
I20260925 10:59:09.326196 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.299759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:35471 (local address 127.0.0.1:58062)
0925 10:59:09.317206 (+ 17447us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:09.317225 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:09.317338 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:09.319729 (+ 2391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:09.319734 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:09.319757 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:09.320185 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:09.320194 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.321564 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.321569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:09.323585 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:09.323598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.325413 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.325419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:09.325590 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.325622 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 10:59:09.325720 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 10:59:09.325843 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17320,"mutex_wait_us":97,"thread_start_us":12788,"threads_started":1}
I20260925 10:59:09.326253 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.299790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58062 (local address 127.31.62.131:35471)
0925 10:59:09.319254 (+ 19464us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:09.319261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:09.319286 (+ 25us) server_negotiation.cc:407] Connection header received
0925 10:59:09.319358 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:09.319363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:09.319552 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:09.319711 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:09.320362 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.321375 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.323748 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.325194 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.325317 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.325539 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:09.325775 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 10:59:09.325952 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19306,"thread_start_us":19163,"threads_started":1}
I20260925 10:59:09.327399 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.301922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:40937 (local address 127.0.0.1:43616)
0925 10:59:09.318144 (+ 16222us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:09.318166 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:09.318304 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:09.319201 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:09.319206 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:09.319235 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:09.319738 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:09.319751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.321618 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.321623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:09.323855 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:09.323868 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.326590 (+ 2722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.326597 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:09.326775 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.326807 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 10:59:09.326896 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 10:59:09.327018 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16097,"mutex_wait_us":156,"thread_start_us":15936,"threads_started":1}
I20260925 10:59:09.327453 17208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.302577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43616 (local address 127.31.62.129:40937)
0925 10:59:09.317687 (+ 15110us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:09.317697 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:09.318409 (+ 712us) server_negotiation.cc:407] Connection header received
0925 10:59:09.318497 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:09.318503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:09.318724 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:09.318947 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:09.319960 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.321480 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.324027 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.325846 (+ 1819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.325966 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.326172 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:09.326963 (+ 791us) server_negotiation.cc:299] Negotiation successful
0925 10:59:09.327131 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14923,"thread_start_us":14740,"threads_started":1}
I20260925 10:59:09.328397 16992 tablet_service.cc:1467] Processing CreateTablet for tablet 2e88f617270c4cd09cbc1331466437de (DEFAULT_TABLE table=test-table [id=e6a5007de92b4590bbeb7d8469f88636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:09.329880 16992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e88f617270c4cd09cbc1331466437de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:09.330217 17126 tablet_service.cc:1467] Processing CreateTablet for tablet 2e88f617270c4cd09cbc1331466437de (DEFAULT_TABLE table=test-table [id=e6a5007de92b4590bbeb7d8469f88636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:09.331471 17126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e88f617270c4cd09cbc1331466437de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:09.336915 16858 tablet_service.cc:1467] Processing CreateTablet for tablet 2e88f617270c4cd09cbc1331466437de (DEFAULT_TABLE table=test-table [id=e6a5007de92b4590bbeb7d8469f88636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:09.338678 16858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e88f617270c4cd09cbc1331466437de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:09.342932 17209 tablet_bootstrap.cc:492] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828: Bootstrap starting.
I20260925 10:59:09.345173 17210 tablet_bootstrap.cc:492] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c: Bootstrap starting.
I20260925 10:59:09.346747 17209 tablet_bootstrap.cc:654] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:09.348093 17209 log.cc:824] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:09.348879 17210 tablet_bootstrap.cc:654] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:09.350222 17210 log.cc:824] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:09.352272 17209 tablet_bootstrap.cc:492] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828: No bootstrap required, opened a new log
I20260925 10:59:09.352595 17209 ts_tablet_manager.cc:1397] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260925 10:59:09.353188 17213 tablet_bootstrap.cc:492] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911: Bootstrap starting.
I20260925 10:59:09.354912 17210 tablet_bootstrap.cc:492] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c: No bootstrap required, opened a new log
I20260925 10:59:09.355340 17210 ts_tablet_manager.cc:1397] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260925 10:59:09.357206 17213 tablet_bootstrap.cc:654] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:09.358748 17213 log.cc:824] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:09.363571 17213 tablet_bootstrap.cc:492] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911: No bootstrap required, opened a new log
I20260925 10:59:09.363512 17209 raft_consensus.cc:348] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35399 } } peers { permanent_uuid: "163d31b82ce64843b09a4e237929b828" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35471 } } peers { permanent_uuid: "a8e21d6a5f834c749472bf82449cf911" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 40937 } }
I20260925 10:59:09.363873 17213 ts_tablet_manager.cc:1397] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260925 10:59:09.364006 17209 raft_consensus.cc:374] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:09.364102 17209 raft_consensus.cc:729] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 163d31b82ce64843b09a4e237929b828, State: Initialized, Role: FOLLOWER
I20260925 10:59:09.364713 17209 consensus_queue.cc:260] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35399 } } peers { permanent_uuid: "163d31b82ce64843b09a4e237929b828" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35471 } } peers { permanent_uuid: "a8e21d6a5f834c749472bf82449cf911" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 40937 } }
I20260925 10:59:09.365782 17191 heartbeater.cc:503] Master 127.31.62.190:42381 was elected leader, sending a full tablet report...
I20260925 10:59:09.367239 17210 raft_consensus.cc:348] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35399 } } peers { permanent_uuid: "163d31b82ce64843b09a4e237929b828" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35471 } } peers { permanent_uuid: "a8e21d6a5f834c749472bf82449cf911" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 40937 } }
I20260925 10:59:09.367699 17210 raft_consensus.cc:374] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:09.367784 17209 ts_tablet_manager.cc:1428] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828: Time spent starting tablet: real 0.015s user 0.008s sys 0.007s
I20260925 10:59:09.367794 17210 raft_consensus.cc:729] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f2eacb05a31414c8a7f08b8fcc01b8c, State: Initialized, Role: FOLLOWER
I20260925 10:59:09.368556 17210 consensus_queue.cc:260] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35399 } } peers { permanent_uuid: "163d31b82ce64843b09a4e237929b828" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35471 } } peers { permanent_uuid: "a8e21d6a5f834c749472bf82449cf911" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 40937 } }
I20260925 10:59:09.369746 17057 heartbeater.cc:503] Master 127.31.62.190:42381 was elected leader, sending a full tablet report...
I20260925 10:59:09.370428 17210 ts_tablet_manager.cc:1428] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260925 10:59:09.373389 17213 raft_consensus.cc:348] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35399 } } peers { permanent_uuid: "163d31b82ce64843b09a4e237929b828" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35471 } } peers { permanent_uuid: "a8e21d6a5f834c749472bf82449cf911" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 40937 } }
I20260925 10:59:09.373759 17213 raft_consensus.cc:374] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:09.373847 17213 raft_consensus.cc:729] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8e21d6a5f834c749472bf82449cf911, State: Initialized, Role: FOLLOWER
I20260925 10:59:09.374415 17213 consensus_queue.cc:260] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35399 } } peers { permanent_uuid: "163d31b82ce64843b09a4e237929b828" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35471 } } peers { permanent_uuid: "a8e21d6a5f834c749472bf82449cf911" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 40937 } }
I20260925 10:59:09.375800 17213 ts_tablet_manager.cc:1428] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
W20260925 10:59:09.449141 16924 tablet.cc:2369] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:59:09.470320 17058 tablet.cc:2369] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:59:09.474663 17192 tablet.cc:2369] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:59:09.507345 17215 raft_consensus.cc:482] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:59:09.507642 17215 raft_consensus.cc:504] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35399 } } peers { permanent_uuid: "163d31b82ce64843b09a4e237929b828" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35471 } } peers { permanent_uuid: "a8e21d6a5f834c749472bf82449cf911" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 40937 } }
I20260925 10:59:09.509467 17215 leader_election.cc:288] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f2eacb05a31414c8a7f08b8fcc01b8c (127.31.62.130:35399), a8e21d6a5f834c749472bf82449cf911 (127.31.62.129:40937)
I20260925 10:59:09.515148 17208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.509884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:38679 (local address 127.31.62.129:40937)
0925 10:59:09.510083 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:09.510089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:09.510383 (+ 294us) server_negotiation.cc:407] Connection header received
0925 10:59:09.510591 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:09.510595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:09.510657 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:09.510731 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:09.511152 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.511874 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.513504 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.514265 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.514411 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.514487 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:09.514775 (+ 288us) server_negotiation.cc:299] Negotiation successful
0925 10:59:09.514909 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 10:59:09.515244 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.509925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:40937 (local address 127.31.62.131:38679)
0925 10:59:09.510270 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:09.510286 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:09.510441 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:09.510764 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:09.510768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:09.510794 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:09.511013 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:09.511022 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.511995 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.512000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:09.513348 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:09.513360 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.514411 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.514417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:09.514617 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.514651 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 10:59:09.514740 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 10:59:09.514902 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":2,"spinlock_wait_cycles":24320}
I20260925 10:59:09.515676 17203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.509890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:49673 (local address 127.31.62.130:35399)
0925 10:59:09.510083 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:09.510090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:09.510836 (+ 746us) server_negotiation.cc:407] Connection header received
0925 10:59:09.511016 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:09.511020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:09.511078 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:09.511150 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:09.511654 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.512338 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.514008 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.514823 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.514967 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.515036 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:09.515335 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 10:59:09.515470 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 10:59:09.516356 17218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.510000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:35399 (local address 127.31.62.131:49673)
0925 10:59:09.510722 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:09.510737 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:09.510881 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:09.511173 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:09.511177 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:09.511200 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:09.511510 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:09.511518 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.512482 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.512486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:09.513857 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:09.513868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.515015 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.515021 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:09.515159 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.515181 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 10:59:09.515889 (+ 708us) client_negotiation.cc:240] Negotiation successful
0925 10:59:09.516015 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":16,"thread_start_us":441,"threads_started":1}
I20260925 10:59:09.516292 16878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e88f617270c4cd09cbc1331466437de" candidate_uuid: "163d31b82ce64843b09a4e237929b828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8e21d6a5f834c749472bf82449cf911" is_pre_election: true
I20260925 10:59:09.516880 16878 raft_consensus.cc:2454] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 163d31b82ce64843b09a4e237929b828 in term 0.
I20260925 10:59:09.517441 17012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e88f617270c4cd09cbc1331466437de" candidate_uuid: "163d31b82ce64843b09a4e237929b828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c" is_pre_election: true
I20260925 10:59:09.517624 17079 leader_election.cc:302] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 163d31b82ce64843b09a4e237929b828, a8e21d6a5f834c749472bf82449cf911; no voters:
I20260925 10:59:09.517948 17012 raft_consensus.cc:2454] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 163d31b82ce64843b09a4e237929b828 in term 0.
I20260925 10:59:09.518035 17215 raft_consensus.cc:2781] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:59:09.518162 17215 raft_consensus.cc:482] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:59:09.518249 17215 raft_consensus.cc:3037] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:09.520377 17215 raft_consensus.cc:504] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35399 } } peers { permanent_uuid: "163d31b82ce64843b09a4e237929b828" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35471 } } peers { permanent_uuid: "a8e21d6a5f834c749472bf82449cf911" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 40937 } }
I20260925 10:59:09.521020 17215 leader_election.cc:288] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [CANDIDATE]: Term 1 election: Requested vote from peers 3f2eacb05a31414c8a7f08b8fcc01b8c (127.31.62.130:35399), a8e21d6a5f834c749472bf82449cf911 (127.31.62.129:40937)
I20260925 10:59:09.521399 17012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e88f617270c4cd09cbc1331466437de" candidate_uuid: "163d31b82ce64843b09a4e237929b828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c"
I20260925 10:59:09.521585 17012 raft_consensus.cc:3037] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:09.521536 16878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e88f617270c4cd09cbc1331466437de" candidate_uuid: "163d31b82ce64843b09a4e237929b828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8e21d6a5f834c749472bf82449cf911"
I20260925 10:59:09.521760 16878 raft_consensus.cc:3037] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:09.523667 17012 raft_consensus.cc:2454] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 163d31b82ce64843b09a4e237929b828 in term 1.
I20260925 10:59:09.523975 16878 raft_consensus.cc:2454] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 163d31b82ce64843b09a4e237929b828 in term 1.
I20260925 10:59:09.524045 17080 leader_election.cc:302] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 163d31b82ce64843b09a4e237929b828, 3f2eacb05a31414c8a7f08b8fcc01b8c; no voters:
I20260925 10:59:09.524374 17215 raft_consensus.cc:2781] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:09.524808 17215 raft_consensus.cc:686] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [term 1 LEADER]: Becoming Leader. State: Replica: 163d31b82ce64843b09a4e237929b828, State: Running, Role: LEADER
I20260925 10:59:09.525274 17215 consensus_queue.cc:237] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35399 } } peers { permanent_uuid: "163d31b82ce64843b09a4e237929b828" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35471 } } peers { permanent_uuid: "a8e21d6a5f834c749472bf82449cf911" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 40937 } }
I20260925 10:59:09.529932 16734 catalog_manager.cc:5495] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 reported cstate change: term changed from 0 to 1, leader changed from <none> to 163d31b82ce64843b09a4e237929b828 (127.31.62.131). New cstate: current_term: 1 leader_uuid: "163d31b82ce64843b09a4e237929b828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "163d31b82ce64843b09a4e237929b828" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 35471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8e21d6a5f834c749472bf82449cf911" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 40937 } health_report { overall_health: UNKNOWN } } }
I20260925 10:59:09.547819 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.542193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58070 (local address 127.31.62.131:35471)
0925 10:59:09.542428 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:09.542433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:09.542564 (+ 131us) server_negotiation.cc:407] Connection header received
0925 10:59:09.542687 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:09.542691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:09.542770 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:09.542897 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:59:09.543280 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.543989 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.545622 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.546013 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.546102 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.546692 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:59:09.547208 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:59:09.547314 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:09.547487 (+ 173us) server_negotiation.cc:299] Negotiation successful
0925 10:59:09.547561 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 10:59:09.556123 16878 raft_consensus.cc:1261] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911 [term 1 FOLLOWER]: Refusing update from remote peer 163d31b82ce64843b09a4e237929b828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:59:09.556124 17012 raft_consensus.cc:1261] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c [term 1 FOLLOWER]: Refusing update from remote peer 163d31b82ce64843b09a4e237929b828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:59:09.557137 17219 consensus_queue.cc:1035] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f2eacb05a31414c8a7f08b8fcc01b8c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 35399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:59:09.557539 17215 consensus_queue.cc:1035] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8e21d6a5f834c749472bf82449cf911" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 40937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:59:09.577003 17222 mvcc.cc:205] Tried to move back new op lower bound from 7333207857371901952 to 7333207857259155456. Current Snapshot: MvccSnapshot[applied={T|T < 7333207857371901952}]
I20260925 10:59:09.578215 17224 mvcc.cc:205] Tried to move back new op lower bound from 7333207857371901952 to 7333207857259155456. Current Snapshot: MvccSnapshot[applied={T|T < 7333207857371901952}]
I20260925 10:59:09.595474 16734 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:59:09.595741 16734 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:59:09.599603 16734 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ae84726585e4c41809cd620b8cc1f61: Sending DeleteTablet for 3 replicas of tablet 2e88f617270c4cd09cbc1331466437de
I20260925 10:59:09.600739 17126 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e88f617270c4cd09cbc1331466437de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:09 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:58062
I20260925 10:59:09.600837 16992 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e88f617270c4cd09cbc1331466437de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:09 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:46992
I20260925 10:59:09.600837 16858 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e88f617270c4cd09cbc1331466437de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:09 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:43616
I20260925 10:59:09.602255 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 16795
I20260925 10:59:09.612550 17234 tablet_replica.cc:330] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911: stopping tablet replica
I20260925 10:59:09.613320 17232 tablet_replica.cc:330] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828: stopping tablet replica
I20260925 10:59:09.613775 17232 raft_consensus.cc:2229] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:59:09.613830 17234 raft_consensus.cc:2229] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:59:09.614279 17234 raft_consensus.cc:2258] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:09.614449 17232 raft_consensus.cc:2258] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:09.615903 17233 tablet_replica.cc:330] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c: stopping tablet replica
I20260925 10:59:09.616359 17233 raft_consensus.cc:2229] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:59:09.616796 17232 ts_tablet_manager.cc:1905] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:09.616792 17234 ts_tablet_manager.cc:1905] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:09.616909 17233 raft_consensus.cc:2258] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:09.619077 17233 ts_tablet_manager.cc:1905] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:09.623950 17234 ts_tablet_manager.cc:1918] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:09.624204 17234 log.cc:1196] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/2e88f617270c4cd09cbc1331466437de
I20260925 10:59:09.624290 17232 ts_tablet_manager.cc:1918] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:09.624480 17232 log.cc:1196] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/2e88f617270c4cd09cbc1331466437de
I20260925 10:59:09.625038 17234 ts_tablet_manager.cc:1939] T 2e88f617270c4cd09cbc1331466437de P a8e21d6a5f834c749472bf82449cf911: Deleting consensus metadata
I20260925 10:59:09.625787 17233 ts_tablet_manager.cc:1918] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:09.625964 17233 log.cc:1196] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/2e88f617270c4cd09cbc1331466437de
I20260925 10:59:09.626468 17233 ts_tablet_manager.cc:1939] T 2e88f617270c4cd09cbc1331466437de P 3f2eacb05a31414c8a7f08b8fcc01b8c: Deleting consensus metadata
I20260925 10:59:09.627370 16718 catalog_manager.cc:4841] TS a8e21d6a5f834c749472bf82449cf911 (127.31.62.129:40937): tablet 2e88f617270c4cd09cbc1331466437de (table test-table [id=e6a5007de92b4590bbeb7d8469f88636]) successfully deleted
I20260925 10:59:09.628149 16719 catalog_manager.cc:4841] TS 3f2eacb05a31414c8a7f08b8fcc01b8c (127.31.62.130:35399): tablet 2e88f617270c4cd09cbc1331466437de (table test-table [id=e6a5007de92b4590bbeb7d8469f88636]) successfully deleted
I20260925 10:59:09.628549 17232 ts_tablet_manager.cc:1939] T 2e88f617270c4cd09cbc1331466437de P 163d31b82ce64843b09a4e237929b828: Deleting consensus metadata
I20260925 10:59:09.630050 16721 catalog_manager.cc:4841] TS 163d31b82ce64843b09a4e237929b828 (127.31.62.131:35471): tablet 2e88f617270c4cd09cbc1331466437de (table test-table [id=e6a5007de92b4590bbeb7d8469f88636]) successfully deleted
Sep 25 10:59:09 dist-test-slave-nx7s krb5kdc[16689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:59:09.641482 17208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.603748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43632 (local address 127.31.62.129:40937)
0925 10:59:09.603942 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:09.603949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:09.609581 (+ 5632us) server_negotiation.cc:407] Connection header received
0925 10:59:09.609830 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:09.609837 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:09.609947 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:09.610055 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:09.625050 (+ 14995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.626226 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.628362 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.629089 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.629193 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.634401 (+ 5208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:09.634418 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:09.634437 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:09.634491 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:09.638214 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:09.639043 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:09.639053 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:09.639056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:09.639231 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:09.639618 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:09.639623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:09.639627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:09.640090 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:09.640388 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:09.640643 (+ 255us) server_negotiation.cc:299] Negotiation successful
0925 10:59:09.641089 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 10:59:09.642221 16918 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:09.742122 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 16795
I20260925 10:59:09.757994 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 16929
Sep 25 10:59:09 dist-test-slave-nx7s krb5kdc[16689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:59:09.775058 17203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.758795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46998 (local address 127.31.62.130:35399)
0925 10:59:09.758990 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:09.758996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:09.759020 (+ 24us) server_negotiation.cc:407] Connection header received
0925 10:59:09.759183 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:09.759189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:09.759272 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:09.759389 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:09.760569 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.761733 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.763267 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.763868 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.764000 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.768146 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:09.768168 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:09.768190 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:09.768237 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:09.771913 (+ 3676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:09.772694 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:09.772704 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:09.772708 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:09.772893 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:09.773275 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:09.773280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:09.773283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:09.773808 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:09.774122 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:09.774442 (+ 320us) server_negotiation.cc:299] Negotiation successful
0925 10:59:09.774684 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 10:59:09.775906 17052 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:09.867733 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 16929
I20260925 10:59:09.888797 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 17063
Sep 25 10:59:09 dist-test-slave-nx7s krb5kdc[16689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:59:09.907627 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:09.889725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58082 (local address 127.31.62.131:35471)
0925 10:59:09.889964 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:09.889968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:09.889986 (+ 18us) server_negotiation.cc:407] Connection header received
0925 10:59:09.890040 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:09.890044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:09.890132 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:09.890213 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:09.891932 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.893044 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.895116 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:09.895598 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:09.895706 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:09.901545 (+ 5839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:09.901565 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:09.901583 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:09.901630 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:09.904611 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:09.905460 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:09.905468 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:09.905470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:09.905629 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:09.906013 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:09.906017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:09.906019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:09.906418 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:09.906712 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:09.907096 (+ 384us) server_negotiation.cc:299] Negotiation successful
0925 10:59:09.907281 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260925 10:59:09.908452 17186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:10.029425 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 17063
I20260925 10:59:10.044924 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 16702
I20260925 10:59:10.046021 16764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:10.157171 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 16702
2026-09-25T10:59:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4321 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-25T10:59:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:10Z Disabled control of system clock
I20260925 10:59:10.231174 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:33861
--webserver_interface=127.31.62.190
--webserver_port=0
--builtin_ntp_servers=127.31.62.148:38747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:33861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 10:59:10.559187 17243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:10.559512 17243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:10.559598 17243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:10.567178 17243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:10.567301 17243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:10.567337 17243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:10.567396 17243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:10.567440 17243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:10.576897 17243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:38747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:33861
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:33861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:10.578168 17243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:10.579882 17243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:10.591312 17248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:10.591439 17249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:10.591467 17251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:10.592391 17243 server_base.cc:1034] running on GCE node
I20260925 10:59:10.593326 17243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:10.594871 17243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:10.596109 17243 hybrid_clock.cc:648] HybridClock initialized: now 1790333950596057 us; error 31 us; skew 500 ppm
I20260925 10:59:10.596508 17243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:59:10.599062 17243 webserver.cc:460] Webserver started at http://127.31.62.190:42401/ using document root <none> and password file <none>
I20260925 10:59:10.599805 17243 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:10.599923 17243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:10.600265 17243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:10.603657 17243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "beb8187dc4f7422dac26402c8cd1cb33"
format_stamp: "Formatted at 2026-09-25 10:59:10 on dist-test-slave-nx7s"
server_key: "089daf69d2c006199afb6efc84b711f5"
server_key_iv: "c25c0fddf3c30cde8852bda1e489252a"
server_key_version: "encryptionkey@0"
I20260925 10:59:10.604641 17243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "beb8187dc4f7422dac26402c8cd1cb33"
format_stamp: "Formatted at 2026-09-25 10:59:10 on dist-test-slave-nx7s"
server_key: "089daf69d2c006199afb6efc84b711f5"
server_key_iv: "c25c0fddf3c30cde8852bda1e489252a"
server_key_version: "encryptionkey@0"
I20260925 10:59:10.610416 17243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 10:59:10.614199 17257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:10.615646 17243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260925 10:59:10.615823 17243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "beb8187dc4f7422dac26402c8cd1cb33"
format_stamp: "Formatted at 2026-09-25 10:59:10 on dist-test-slave-nx7s"
server_key: "089daf69d2c006199afb6efc84b711f5"
server_key_iv: "c25c0fddf3c30cde8852bda1e489252a"
server_key_version: "encryptionkey@0"
I20260925 10:59:10.616026 17243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:10.628415 17243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:10.629320 17243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:10.629597 17243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:10.647178 17243 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:33861
I20260925 10:59:10.647198 17308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:33861 every 8 connection(s)
I20260925 10:59:10.648499 17243 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:10.652354 17309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:10.654809 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 17243
I20260925 10:59:10.654983 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:10.655534 31994 external_mini_cluster.cc:1428] Setting key 22b78543f8ea2c33b0d144d6ae9d3bdf
I20260925 10:59:10.664757 17309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33: Bootstrap starting.
I20260925 10:59:10.668347 17309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:10.669718 17309 log.cc:824] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:10.670467 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:10.657558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44964 (local address 127.31.62.190:33861)
0925 10:59:10.658678 (+ 1120us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:10.658690 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:10.658733 (+ 43us) server_negotiation.cc:407] Connection header received
0925 10:59:10.660457 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:10.660482 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:10.661055 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:10.661423 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:10.662099 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:10.663510 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:10.665410 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:10.666183 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:10.666351 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:10.667546 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:10.667554 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:10.667569 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:10.668110 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:10.668201 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:10.668222 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:10.668309 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:10.668657 (+ 348us) server_negotiation.cc:299] Negotiation successful
0925 10:59:10.669082 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":411,"threads_started":1}
I20260925 10:59:10.672673 17309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33: No bootstrap required, opened a new log
I20260925 10:59:10.680276 17309 raft_consensus.cc:348] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb8187dc4f7422dac26402c8cd1cb33" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 33861 } }
I20260925 10:59:10.680687 17309 raft_consensus.cc:374] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:10.680989 17309 raft_consensus.cc:729] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb8187dc4f7422dac26402c8cd1cb33, State: Initialized, Role: FOLLOWER
I20260925 10:59:10.681674 17309 consensus_queue.cc:260] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb8187dc4f7422dac26402c8cd1cb33" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 33861 } }
I20260925 10:59:10.681924 17309 raft_consensus.cc:388] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:10.682067 17309 raft_consensus.cc:482] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:10.682231 17309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:10.684557 17309 raft_consensus.cc:504] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb8187dc4f7422dac26402c8cd1cb33" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 33861 } }
I20260925 10:59:10.685109 17309 leader_election.cc:302] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beb8187dc4f7422dac26402c8cd1cb33; no voters:
I20260925 10:59:10.685720 17309 leader_election.cc:288] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:10.685946 17314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:10.686601 17314 raft_consensus.cc:686] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [term 1 LEADER]: Becoming Leader. State: Replica: beb8187dc4f7422dac26402c8cd1cb33, State: Running, Role: LEADER
I20260925 10:59:10.687131 17314 consensus_queue.cc:237] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb8187dc4f7422dac26402c8cd1cb33" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 33861 } }
I20260925 10:59:10.687819 17309 sys_catalog.cc:564] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:10.689767 17315 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "beb8187dc4f7422dac26402c8cd1cb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb8187dc4f7422dac26402c8cd1cb33" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 33861 } } }
I20260925 10:59:10.690200 17315 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:10.690879 17316 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader beb8187dc4f7422dac26402c8cd1cb33. Latest consensus state: current_term: 1 leader_uuid: "beb8187dc4f7422dac26402c8cd1cb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb8187dc4f7422dac26402c8cd1cb33" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 33861 } } }
I20260925 10:59:10.691229 17316 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:10.691542 17322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:10.696590 17322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:10.704932 17322 catalog_manager.cc:1347] Generated new cluster ID: be18fe42ef414ccfba0102e41f0d00f7
I20260925 10:59:10.705060 17322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:10.729789 17322 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:10.731096 17322 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:10.744567 17322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33: Generated new TSK 0
I20260925 10:59:10.745432 17322 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:10.764288 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:33861
--builtin_ntp_servers=127.31.62.148:38747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 10:59:11.095018 17333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:11.095299 17333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:11.095422 17333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:11.102619 17333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:11.102728 17333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:11.102873 17333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:59:11.111975 17333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:38747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:33861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:11.113329 17333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:11.115196 17333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:11.128254 17338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:11.128214 17339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:11.128185 17341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:11.129055 17333 server_base.cc:1034] running on GCE node
I20260925 10:59:11.129652 17333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:11.130612 17333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:11.131856 17333 hybrid_clock.cc:648] HybridClock initialized: now 1790333951131786 us; error 53 us; skew 500 ppm
I20260925 10:59:11.132220 17333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:59:11.134807 17333 webserver.cc:460] Webserver started at http://127.31.62.129:40747/ using document root <none> and password file <none>
I20260925 10:59:11.135582 17333 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:11.135689 17333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:11.136024 17333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:11.139238 17333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "28beec99d7b84ef382620323518bf02f"
format_stamp: "Formatted at 2026-09-25 10:59:11 on dist-test-slave-nx7s"
server_key: "db35987390c40a22b2830e7becd9fc64"
server_key_iv: "9422ce2bf230bb8c32f1b5bc146cdfa5"
server_key_version: "encryptionkey@0"
I20260925 10:59:11.140082 17333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "28beec99d7b84ef382620323518bf02f"
format_stamp: "Formatted at 2026-09-25 10:59:11 on dist-test-slave-nx7s"
server_key: "db35987390c40a22b2830e7becd9fc64"
server_key_iv: "9422ce2bf230bb8c32f1b5bc146cdfa5"
server_key_version: "encryptionkey@0"
I20260925 10:59:11.145685 17333 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 10:59:11.149806 17347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:11.151186 17333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260925 10:59:11.151340 17333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "28beec99d7b84ef382620323518bf02f"
format_stamp: "Formatted at 2026-09-25 10:59:11 on dist-test-slave-nx7s"
server_key: "db35987390c40a22b2830e7becd9fc64"
server_key_iv: "9422ce2bf230bb8c32f1b5bc146cdfa5"
server_key_version: "encryptionkey@0"
I20260925 10:59:11.151541 17333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:11.185660 17333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:11.186568 17333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:11.186862 17333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:11.187870 17333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:11.189508 17333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:11.189606 17333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:11.189707 17333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:11.189774 17333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:11.220402 17333 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:38073
I20260925 10:59:11.220500 17460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:38073 every 8 connection(s)
I20260925 10:59:11.221798 17333 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:59:11.227454 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 17333
I20260925 10:59:11.227626 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:59:11.228089 31994 external_mini_cluster.cc:1428] Setting key f11fb259baee200898a92451c6f3d64e
I20260925 10:59:11.231149 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:33861
--builtin_ntp_servers=127.31.62.148:38747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 10:59:11.235553 17463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:11.224337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:54585 (local address 127.31.62.190:33861)
0925 10:59:11.224779 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:11.224784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:11.226044 (+ 1260us) server_negotiation.cc:407] Connection header received
0925 10:59:11.227960 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:11.227966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:11.228056 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:11.228153 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:11.229616 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:11.230677 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:11.232476 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:11.233088 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:11.233211 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:11.234584 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:11.234591 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:11.234595 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:11.234619 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:11.234664 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:11.234677 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:11.234771 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:11.235102 (+ 331us) server_negotiation.cc:299] Negotiation successful
0925 10:59:11.235203 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":241,"threads_started":1}
I20260925 10:59:11.236693 17464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:11.224901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:33861 (local address 127.31.62.129:54585)
0925 10:59:11.225818 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:11.225920 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:11.227624 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:11.228435 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:11.228444 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:11.228551 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:11.229427 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:11.229455 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:11.230916 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:11.230920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:11.232306 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:11.232324 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:11.233324 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:11.233331 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:11.233569 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:11.234260 (+ 691us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:59:11.234263 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:11.234311 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:59:11.234402 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:11.234785 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:11.234795 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:11.234802 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 10:59:11.235077 (+ 275us) client_negotiation.cc:240] Negotiation successful
0925 10:59:11.235466 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":250,"threads_started":1}
I20260925 10:59:11.238317 17461 heartbeater.cc:347] Connected to a master server at 127.31.62.190:33861
I20260925 10:59:11.238613 17461 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:11.239384 17461 heartbeater.cc:511] Master 127.31.62.190:33861 requested a full tablet report, sending...
I20260925 10:59:11.241606 17274 ts_manager.cc:194] Registered new tserver with Master: 28beec99d7b84ef382620323518bf02f (127.31.62.129:38073)
I20260925 10:59:11.243145 17274 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.62.129:54585
W20260925 10:59:11.559695 17465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:11.560002 17465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:11.560110 17465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:11.567408 17465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:11.567514 17465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:11.567651 17465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:59:11.577175 17465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:38747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:33861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:11.578488 17465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:11.580331 17465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:11.593321 17471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:11.593328 17473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:11.593493 17470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:11.594156 17465 server_base.cc:1034] running on GCE node
I20260925 10:59:11.594785 17465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:11.595750 17465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:11.597011 17465 hybrid_clock.cc:648] HybridClock initialized: now 1790333951596964 us; error 42 us; skew 500 ppm
I20260925 10:59:11.597343 17465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:59:11.599612 17465 webserver.cc:460] Webserver started at http://127.31.62.130:45957/ using document root <none> and password file <none>
I20260925 10:59:11.600297 17465 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:11.600407 17465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:11.600713 17465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:11.603649 17465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "f2cc970f48b04b22a22949aa3190b89e"
format_stamp: "Formatted at 2026-09-25 10:59:11 on dist-test-slave-nx7s"
server_key: "89fcf7f3a153fb89997ce3d9085028d0"
server_key_iv: "25d2def50d9d4f5d083e62e18bd5ef26"
server_key_version: "encryptionkey@0"
I20260925 10:59:11.604430 17465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "f2cc970f48b04b22a22949aa3190b89e"
format_stamp: "Formatted at 2026-09-25 10:59:11 on dist-test-slave-nx7s"
server_key: "89fcf7f3a153fb89997ce3d9085028d0"
server_key_iv: "25d2def50d9d4f5d083e62e18bd5ef26"
server_key_version: "encryptionkey@0"
I20260925 10:59:11.609531 17465 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260925 10:59:11.613065 17479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:11.614513 17465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:59:11.614677 17465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "f2cc970f48b04b22a22949aa3190b89e"
format_stamp: "Formatted at 2026-09-25 10:59:11 on dist-test-slave-nx7s"
server_key: "89fcf7f3a153fb89997ce3d9085028d0"
server_key_iv: "25d2def50d9d4f5d083e62e18bd5ef26"
server_key_version: "encryptionkey@0"
I20260925 10:59:11.614880 17465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:11.639617 17465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:11.640507 17465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:11.640800 17465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:11.641868 17465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:11.643445 17465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:11.643539 17465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:11.643644 17465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:11.643714 17465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:11.680095 17465 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:41023
I20260925 10:59:11.680184 17591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:41023 every 8 connection(s)
I20260925 10:59:11.681439 17465 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:59:11.686285 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 17465
I20260925 10:59:11.686446 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:59:11.686914 31994 external_mini_cluster.cc:1428] Setting key a3d6ddd98b79d1a3b356c9f3227a02fa
I20260925 10:59:11.689736 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:33861
--builtin_ntp_servers=127.31.62.148:38747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 10:59:11.694303 17463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:11.684083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:45505 (local address 127.31.62.190:33861)
0925 10:59:11.684262 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:11.684267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:11.685727 (+ 1460us) server_negotiation.cc:407] Connection header received
0925 10:59:11.687694 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:11.687700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:11.687814 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:11.687935 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:11.689271 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:11.690042 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:11.691596 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:11.691997 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:11.692107 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:11.693512 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:11.693520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:11.693523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:11.693541 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:11.693572 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:11.693604 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:11.693680 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:11.693902 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 10:59:11.694003 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 10:59:11.695452 17594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:11.684627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:33861 (local address 127.31.62.130:45505)
0925 10:59:11.685560 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:11.685595 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:11.687363 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:11.688209 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:11.688219 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:11.688321 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:11.689112 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:11.689133 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:11.690191 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:11.690197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:11.691458 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:11.691471 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:11.692225 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:11.692232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:11.692462 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:11.693194 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:59:11.693197 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:11.693237 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:59:11.693325 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:11.693692 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:11.693703 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:11.693711 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 10:59:11.693862 (+ 151us) client_negotiation.cc:240] Negotiation successful
0925 10:59:11.694308 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":303,"threads_started":1}
I20260925 10:59:11.696935 17592 heartbeater.cc:347] Connected to a master server at 127.31.62.190:33861
I20260925 10:59:11.697223 17592 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:11.697958 17592 heartbeater.cc:511] Master 127.31.62.190:33861 requested a full tablet report, sending...
I20260925 10:59:11.699473 17274 ts_manager.cc:194] Registered new tserver with Master: f2cc970f48b04b22a22949aa3190b89e (127.31.62.130:41023)
I20260925 10:59:11.700433 17274 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.62.130:45505
W20260925 10:59:12.010190 17595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:12.010486 17595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:12.010571 17595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:12.017794 17595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:12.017899 17595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:12.018033 17595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:59:12.027293 17595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:38747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:33861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:12.028509 17595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:12.030195 17595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:12.042522 17603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:12.042603 17601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:12.042639 17600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:12.043426 17595 server_base.cc:1034] running on GCE node
I20260925 10:59:12.044066 17595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:12.045115 17595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:12.046337 17595 hybrid_clock.cc:648] HybridClock initialized: now 1790333952046262 us; error 69 us; skew 500 ppm
I20260925 10:59:12.046674 17595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:59:12.049057 17595 webserver.cc:460] Webserver started at http://127.31.62.131:45197/ using document root <none> and password file <none>
I20260925 10:59:12.049798 17595 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:12.049906 17595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:12.050227 17595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:12.053133 17595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "127af10ca57a4378b91abea332b7fff1"
format_stamp: "Formatted at 2026-09-25 10:59:12 on dist-test-slave-nx7s"
server_key: "cc921a1639a9cd7a9ffa1a02f51c5262"
server_key_iv: "8fe2184dad613f01497115c32a11fb6a"
server_key_version: "encryptionkey@0"
I20260925 10:59:12.053893 17595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "127af10ca57a4378b91abea332b7fff1"
format_stamp: "Formatted at 2026-09-25 10:59:12 on dist-test-slave-nx7s"
server_key: "cc921a1639a9cd7a9ffa1a02f51c5262"
server_key_iv: "8fe2184dad613f01497115c32a11fb6a"
server_key_version: "encryptionkey@0"
I20260925 10:59:12.059124 17595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260925 10:59:12.062592 17609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:12.063913 17595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:59:12.064069 17595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "127af10ca57a4378b91abea332b7fff1"
format_stamp: "Formatted at 2026-09-25 10:59:12 on dist-test-slave-nx7s"
server_key: "cc921a1639a9cd7a9ffa1a02f51c5262"
server_key_iv: "8fe2184dad613f01497115c32a11fb6a"
server_key_version: "encryptionkey@0"
I20260925 10:59:12.064270 17595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:12.075703 17595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:12.076551 17595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:12.076918 17595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:12.078016 17595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:12.079391 17595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:12.079480 17595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:12.079589 17595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:12.079658 17595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:12.109694 17595 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:34179
I20260925 10:59:12.109742 17721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:34179 every 8 connection(s)
I20260925 10:59:12.110949 17595 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:59:12.113588 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 17595
I20260925 10:59:12.113754 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:59:12.114188 31994 external_mini_cluster.cc:1428] Setting key e6b8303c1383e750b5d03028df367848
I20260925 10:59:12.123028 17463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.113372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:33525 (local address 127.31.62.190:33861)
0925 10:59:12.113572 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:12.113579 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:12.115088 (+ 1509us) server_negotiation.cc:407] Connection header received
0925 10:59:12.117027 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:12.117031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:12.117097 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:12.117179 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:12.118211 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.118976 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.120609 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.121043 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.121138 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.122330 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:12.122336 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:12.122339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:12.122357 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:12.122395 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:12.122407 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:12.122473 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:12.122643 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 10:59:12.122719 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 10:59:12.123926 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.113918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:33861 (local address 127.31.62.131:33525)
0925 10:59:12.114932 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:12.114981 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:12.116757 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:12.117374 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:12.117383 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:12.117474 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:12.118061 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:12.118078 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.119108 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.119112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:12.120497 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:12.120509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.121217 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.121221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:12.121414 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.122034 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:59:12.122036 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:12.122076 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:59:12.122156 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:12.122485 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:12.122492 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:12.122497 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 10:59:12.122631 (+ 134us) client_negotiation.cc:240] Negotiation successful
0925 10:59:12.122921 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":300,"threads_started":1}
I20260925 10:59:12.125243 17722 heartbeater.cc:347] Connected to a master server at 127.31.62.190:33861
I20260925 10:59:12.125524 17722 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:12.126271 17722 heartbeater.cc:511] Master 127.31.62.190:33861 requested a full tablet report, sending...
I20260925 10:59:12.127709 17274 ts_manager.cc:194] Registered new tserver with Master: 127af10ca57a4378b91abea332b7fff1 (127.31.62.131:34179)
I20260925 10:59:12.128633 17274 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.62.131:33525
I20260925 10:59:12.129242 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:59:12.138815 17463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.133140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44972 (local address 127.31.62.190:33861)
0925 10:59:12.133289 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:12.133295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:12.133680 (+ 385us) server_negotiation.cc:407] Connection header received
0925 10:59:12.133844 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:12.133850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:12.133959 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:12.134040 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:12.134554 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.135231 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.136576 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.137029 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.137137 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.138170 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:12.138176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:12.138179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:12.138196 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:12.138229 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:12.138240 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:12.138324 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:12.138447 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 10:59:12.138542 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 10:59:12.142545 17274 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:59:12.149741 17274 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:59:12.187938 17737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.162693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:41023 (local address 127.0.0.1:35478)
0925 10:59:12.178169 (+ 15476us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:12.178231 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:12.178498 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:12.180790 (+ 2292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:12.180797 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:12.180823 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:12.181322 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:12.181337 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.182949 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.182954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:12.185206 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:12.185219 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.186970 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.186976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:12.187181 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.187234 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 10:59:12.187373 (+ 139us) client_negotiation.cc:240] Negotiation successful
0925 10:59:12.187569 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15126,"mutex_wait_us":106,"thread_start_us":16307,"threads_started":1}
I20260925 10:59:12.188968 17735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.162936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35478 (local address 127.31.62.130:41023)
0925 10:59:12.179937 (+ 17001us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:12.179954 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:12.179984 (+ 30us) server_negotiation.cc:407] Connection header received
0925 10:59:12.180086 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:12.180094 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:12.180403 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:12.180621 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:12.181559 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.182776 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.185392 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.186352 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.186501 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.186846 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:12.188378 (+ 1532us) server_negotiation.cc:299] Negotiation successful
0925 10:59:12.188583 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16800,"thread_start_us":16682,"threads_started":1}
I20260925 10:59:12.189325 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.163896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34370 (local address 127.31.62.131:34179)
0925 10:59:12.172412 (+ 8516us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:12.172423 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:12.180490 (+ 8067us) server_negotiation.cc:407] Connection header received
0925 10:59:12.180804 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:12.180812 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:12.181044 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:12.181215 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:12.181818 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.183465 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.186457 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.187826 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.187953 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.188223 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:12.188776 (+ 553us) server_negotiation.cc:299] Negotiation successful
0925 10:59:12.188975 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8282,"thread_start_us":8110,"threads_started":1}
I20260925 10:59:12.189440 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.163083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:38073 (local address 127.0.0.1:43488)
0925 10:59:12.179452 (+ 16369us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:12.179479 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:12.180432 (+ 953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:12.181453 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:12.181458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:12.181482 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:12.181913 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:12.181925 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.183672 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.183676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:12.186205 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:12.186222 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.188600 (+ 2378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.188607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:12.188788 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.188817 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 10:59:12.188939 (+ 122us) client_negotiation.cc:240] Negotiation successful
0925 10:59:12.189064 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16111,"mutex_wait_us":224,"thread_start_us":16956,"threads_started":1}
I20260925 10:59:12.189733 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.163281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:34179 (local address 127.0.0.1:34370)
0925 10:59:12.180299 (+ 17018us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:12.180316 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:12.180611 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:12.181223 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:12.181227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:12.181256 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:12.181650 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:12.181662 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.183638 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.183643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:12.186242 (+ 2599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:12.186252 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.188015 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.188022 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:12.188186 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.188217 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 10:59:12.188319 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 10:59:12.188468 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16904,"mutex_wait_us":166,"thread_start_us":15460,"threads_started":1}
I20260925 10:59:12.189957 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.163706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43488 (local address 127.31.62.129:38073)
0925 10:59:12.174071 (+ 10365us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:12.174082 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:12.179630 (+ 5548us) server_negotiation.cc:407] Connection header received
0925 10:59:12.180670 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:12.180680 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:12.180988 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:12.181166 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:12.182097 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.183530 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.186646 (+ 3116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.188070 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.188190 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.188476 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:12.189445 (+ 969us) server_negotiation.cc:299] Negotiation successful
0925 10:59:12.189625 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10163,"thread_start_us":10194,"threads_started":1}
I20260925 10:59:12.193662 17396 tablet_service.cc:1467] Processing CreateTablet for tablet b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 (DEFAULT_TABLE table=test-table [id=b560e3a69fb04fe5878f5f407195d73f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:12.193678 17657 tablet_service.cc:1467] Processing CreateTablet for tablet b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 (DEFAULT_TABLE table=test-table [id=b560e3a69fb04fe5878f5f407195d73f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:12.193679 17527 tablet_service.cc:1467] Processing CreateTablet for tablet b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 (DEFAULT_TABLE table=test-table [id=b560e3a69fb04fe5878f5f407195d73f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:12.195106 17396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0282f5e5b9b4cd7a0a1a8f2ce1af0a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:12.195181 17657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0282f5e5b9b4cd7a0a1a8f2ce1af0a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:12.195181 17527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0282f5e5b9b4cd7a0a1a8f2ce1af0a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:12.207945 17740 tablet_bootstrap.cc:492] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f: Bootstrap starting.
I20260925 10:59:12.211266 17740 tablet_bootstrap.cc:654] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:12.212317 17740 log.cc:824] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:12.215348 17742 tablet_bootstrap.cc:492] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e: Bootstrap starting.
I20260925 10:59:12.215606 17740 tablet_bootstrap.cc:492] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f: No bootstrap required, opened a new log
I20260925 10:59:12.215948 17740 ts_tablet_manager.cc:1397] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260925 10:59:12.216272 17741 tablet_bootstrap.cc:492] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1: Bootstrap starting.
I20260925 10:59:12.219389 17742 tablet_bootstrap.cc:654] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:12.219923 17741 tablet_bootstrap.cc:654] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:12.220691 17742 log.cc:824] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:12.221436 17741 log.cc:824] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:12.227065 17740 raft_consensus.cc:348] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2cc970f48b04b22a22949aa3190b89e" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41023 } } peers { permanent_uuid: "28beec99d7b84ef382620323518bf02f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 38073 } } peers { permanent_uuid: "127af10ca57a4378b91abea332b7fff1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34179 } }
I20260925 10:59:12.227595 17740 raft_consensus.cc:374] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:12.227707 17740 raft_consensus.cc:729] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28beec99d7b84ef382620323518bf02f, State: Initialized, Role: FOLLOWER
I20260925 10:59:12.227867 17742 tablet_bootstrap.cc:492] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e: No bootstrap required, opened a new log
I20260925 10:59:12.227876 17741 tablet_bootstrap.cc:492] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1: No bootstrap required, opened a new log
I20260925 10:59:12.228181 17741 ts_tablet_manager.cc:1397] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260925 10:59:12.228180 17742 ts_tablet_manager.cc:1397] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260925 10:59:12.228520 17740 consensus_queue.cc:260] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2cc970f48b04b22a22949aa3190b89e" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41023 } } peers { permanent_uuid: "28beec99d7b84ef382620323518bf02f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 38073 } } peers { permanent_uuid: "127af10ca57a4378b91abea332b7fff1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34179 } }
I20260925 10:59:12.239063 17741 raft_consensus.cc:348] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2cc970f48b04b22a22949aa3190b89e" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41023 } } peers { permanent_uuid: "28beec99d7b84ef382620323518bf02f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 38073 } } peers { permanent_uuid: "127af10ca57a4378b91abea332b7fff1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34179 } }
I20260925 10:59:12.239686 17741 raft_consensus.cc:374] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:12.239836 17741 raft_consensus.cc:729] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 127af10ca57a4378b91abea332b7fff1, State: Initialized, Role: FOLLOWER
I20260925 10:59:12.239686 17742 raft_consensus.cc:348] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2cc970f48b04b22a22949aa3190b89e" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41023 } } peers { permanent_uuid: "28beec99d7b84ef382620323518bf02f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 38073 } } peers { permanent_uuid: "127af10ca57a4378b91abea332b7fff1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34179 } }
I20260925 10:59:12.240160 17742 raft_consensus.cc:374] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:12.240288 17742 raft_consensus.cc:729] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2cc970f48b04b22a22949aa3190b89e, State: Initialized, Role: FOLLOWER
I20260925 10:59:12.240796 17741 consensus_queue.cc:260] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2cc970f48b04b22a22949aa3190b89e" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41023 } } peers { permanent_uuid: "28beec99d7b84ef382620323518bf02f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 38073 } } peers { permanent_uuid: "127af10ca57a4378b91abea332b7fff1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34179 } }
I20260925 10:59:12.241032 17742 consensus_queue.cc:260] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2cc970f48b04b22a22949aa3190b89e" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41023 } } peers { permanent_uuid: "28beec99d7b84ef382620323518bf02f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 38073 } } peers { permanent_uuid: "127af10ca57a4378b91abea332b7fff1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34179 } }
I20260925 10:59:12.245558 17461 heartbeater.cc:503] Master 127.31.62.190:33861 was elected leader, sending a full tablet report...
I20260925 10:59:12.245934 17722 heartbeater.cc:503] Master 127.31.62.190:33861 was elected leader, sending a full tablet report...
I20260925 10:59:12.246344 17592 heartbeater.cc:503] Master 127.31.62.190:33861 was elected leader, sending a full tablet report...
I20260925 10:59:12.248126 17742 ts_tablet_manager.cc:1428] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e: Time spent starting tablet: real 0.020s user 0.017s sys 0.000s
I20260925 10:59:12.248400 17740 ts_tablet_manager.cc:1428] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f: Time spent starting tablet: real 0.032s user 0.024s sys 0.008s
I20260925 10:59:12.250142 17741 ts_tablet_manager.cc:1428] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1: Time spent starting tablet: real 0.022s user 0.013s sys 0.003s
W20260925 10:59:12.362767 17723 tablet.cc:2369] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:59:12.433604 17593 tablet.cc:2369] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:59:12.474265 17462 tablet.cc:2369] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:59:12.571861 17747 raft_consensus.cc:482] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:59:12.572165 17747 raft_consensus.cc:504] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2cc970f48b04b22a22949aa3190b89e" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41023 } } peers { permanent_uuid: "28beec99d7b84ef382620323518bf02f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 38073 } } peers { permanent_uuid: "127af10ca57a4378b91abea332b7fff1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34179 } }
I20260925 10:59:12.573946 17747 leader_election.cc:288] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2cc970f48b04b22a22949aa3190b89e (127.31.62.130:41023), 28beec99d7b84ef382620323518bf02f (127.31.62.129:38073)
I20260925 10:59:12.580336 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.574501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:41023 (local address 127.31.62.131:44263)
0925 10:59:12.574781 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:12.574802 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:12.574975 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:12.575372 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:12.575375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:12.575435 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:12.575644 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:12.575657 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.576929 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.576935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:12.578422 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:12.578434 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.579645 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.579652 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:12.579815 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.579849 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 10:59:12.579931 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 10:59:12.580081 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":1}
I20260925 10:59:12.580365 17735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.574376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:44263 (local address 127.31.62.130:41023)
0925 10:59:12.574586 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:12.574592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:12.574915 (+ 323us) server_negotiation.cc:407] Connection header received
0925 10:59:12.575139 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:12.575147 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:12.575237 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:12.575347 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:12.575822 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.576756 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.578580 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.579490 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.579618 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.579672 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:12.579963 (+ 291us) server_negotiation.cc:299] Negotiation successful
0925 10:59:12.580088 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 10:59:12.580698 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.574513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:35153 (local address 127.31.62.129:38073)
0925 10:59:12.574725 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:12.574733 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:12.575302 (+ 569us) server_negotiation.cc:407] Connection header received
0925 10:59:12.575471 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:12.575476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:12.575566 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:12.575664 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:12.576300 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.577109 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.578971 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.579806 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.579966 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.580027 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:12.580336 (+ 309us) server_negotiation.cc:299] Negotiation successful
0925 10:59:12.580462 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 10:59:12.580770 17749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.574518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:38073 (local address 127.31.62.131:35153)
0925 10:59:12.575192 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:12.575210 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:12.575345 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:12.575702 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:12.575705 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:12.575726 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:12.576126 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:12.576139 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.577258 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.577263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:12.578726 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:12.578740 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.579988 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.579994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:12.580169 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.580206 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 10:59:12.580297 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 10:59:12.580426 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":97,"thread_start_us":307,"threads_started":1}
I20260925 10:59:12.581533 17547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0282f5e5b9b4cd7a0a1a8f2ce1af0a0" candidate_uuid: "127af10ca57a4378b91abea332b7fff1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2cc970f48b04b22a22949aa3190b89e" is_pre_election: true
I20260925 10:59:12.581789 17416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0282f5e5b9b4cd7a0a1a8f2ce1af0a0" candidate_uuid: "127af10ca57a4378b91abea332b7fff1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28beec99d7b84ef382620323518bf02f" is_pre_election: true
I20260925 10:59:12.582034 17547 raft_consensus.cc:2454] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 127af10ca57a4378b91abea332b7fff1 in term 0.
I20260925 10:59:12.582261 17416 raft_consensus.cc:2454] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 127af10ca57a4378b91abea332b7fff1 in term 0.
I20260925 10:59:12.582753 17611 leader_election.cc:302] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 127af10ca57a4378b91abea332b7fff1, f2cc970f48b04b22a22949aa3190b89e; no voters:
I20260925 10:59:12.583160 17747 raft_consensus.cc:2781] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:59:12.583287 17747 raft_consensus.cc:482] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:59:12.583402 17747 raft_consensus.cc:3037] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:12.585606 17747 raft_consensus.cc:504] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2cc970f48b04b22a22949aa3190b89e" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41023 } } peers { permanent_uuid: "28beec99d7b84ef382620323518bf02f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 38073 } } peers { permanent_uuid: "127af10ca57a4378b91abea332b7fff1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34179 } }
I20260925 10:59:12.586162 17747 leader_election.cc:288] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [CANDIDATE]: Term 1 election: Requested vote from peers f2cc970f48b04b22a22949aa3190b89e (127.31.62.130:41023), 28beec99d7b84ef382620323518bf02f (127.31.62.129:38073)
I20260925 10:59:12.586632 17547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0282f5e5b9b4cd7a0a1a8f2ce1af0a0" candidate_uuid: "127af10ca57a4378b91abea332b7fff1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2cc970f48b04b22a22949aa3190b89e"
I20260925 10:59:12.586635 17416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0282f5e5b9b4cd7a0a1a8f2ce1af0a0" candidate_uuid: "127af10ca57a4378b91abea332b7fff1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28beec99d7b84ef382620323518bf02f"
I20260925 10:59:12.586823 17547 raft_consensus.cc:3037] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:12.586835 17416 raft_consensus.cc:3037] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:12.589165 17416 raft_consensus.cc:2454] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 127af10ca57a4378b91abea332b7fff1 in term 1.
I20260925 10:59:12.589210 17547 raft_consensus.cc:2454] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 127af10ca57a4378b91abea332b7fff1 in term 1.
I20260925 10:59:12.589658 17611 leader_election.cc:302] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 127af10ca57a4378b91abea332b7fff1, f2cc970f48b04b22a22949aa3190b89e; no voters:
I20260925 10:59:12.589936 17747 raft_consensus.cc:2781] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:12.590366 17747 raft_consensus.cc:686] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [term 1 LEADER]: Becoming Leader. State: Replica: 127af10ca57a4378b91abea332b7fff1, State: Running, Role: LEADER
I20260925 10:59:12.590883 17747 consensus_queue.cc:237] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2cc970f48b04b22a22949aa3190b89e" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41023 } } peers { permanent_uuid: "28beec99d7b84ef382620323518bf02f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 38073 } } peers { permanent_uuid: "127af10ca57a4378b91abea332b7fff1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34179 } }
I20260925 10:59:12.595552 17273 catalog_manager.cc:5495] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 127af10ca57a4378b91abea332b7fff1 (127.31.62.131). New cstate: current_term: 1 leader_uuid: "127af10ca57a4378b91abea332b7fff1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2cc970f48b04b22a22949aa3190b89e" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28beec99d7b84ef382620323518bf02f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 38073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "127af10ca57a4378b91abea332b7fff1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 34179 } health_report { overall_health: HEALTHY } } }
I20260925 10:59:12.651328 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.645116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34378 (local address 127.31.62.131:34179)
0925 10:59:12.645288 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:12.645293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:12.645645 (+ 352us) server_negotiation.cc:407] Connection header received
0925 10:59:12.645855 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:12.645859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:12.645961 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:12.646084 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:59:12.646440 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.647143 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.649214 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.649648 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.649761 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.650289 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:59:12.650835 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:59:12.650942 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:12.651051 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 10:59:12.651119 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 10:59:12.660678 17547 raft_consensus.cc:1261] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e [term 1 FOLLOWER]: Refusing update from remote peer 127af10ca57a4378b91abea332b7fff1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:59:12.660887 17416 raft_consensus.cc:1261] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f [term 1 FOLLOWER]: Refusing update from remote peer 127af10ca57a4378b91abea332b7fff1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:59:12.661876 17747 consensus_queue.cc:1035] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2cc970f48b04b22a22949aa3190b89e" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 41023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:59:12.662187 17750 consensus_queue.cc:1035] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28beec99d7b84ef382620323518bf02f" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 38073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:59:12.689774 17271 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:59:12.690122 17271 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:59:12.694284 17271 catalog_manager.cc:5782] T 00000000000000000000000000000000 P beb8187dc4f7422dac26402c8cd1cb33: Sending DeleteTablet for 3 replicas of tablet b0282f5e5b9b4cd7a0a1a8f2ce1af0a0
I20260925 10:59:12.695541 17396 tablet_service.cc:1514] Processing DeleteTablet for tablet b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:12 UTC) from {username='slave'} at 127.0.0.1:43488
I20260925 10:59:12.695541 17527 tablet_service.cc:1514] Processing DeleteTablet for tablet b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:12 UTC) from {username='slave'} at 127.0.0.1:35478
I20260925 10:59:12.695591 17657 tablet_service.cc:1514] Processing DeleteTablet for tablet b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:12 UTC) from {username='slave'} at 127.0.0.1:34370
I20260925 10:59:12.697721 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 17333
I20260925 10:59:12.713812 17766 tablet_replica.cc:330] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1: stopping tablet replica
I20260925 10:59:12.714332 17766 raft_consensus.cc:2229] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:59:12.714999 17766 raft_consensus.cc:2258] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:12.715895 17764 tablet_replica.cc:330] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e: stopping tablet replica
I20260925 10:59:12.716365 17764 raft_consensus.cc:2229] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:59:12.717365 17765 tablet_replica.cc:330] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f: stopping tablet replica
I20260925 10:59:12.717653 17764 raft_consensus.cc:2258] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:12.717864 17765 raft_consensus.cc:2229] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:59:12.718461 17765 raft_consensus.cc:2258] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:12.719462 17766 ts_tablet_manager.cc:1905] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:12.719789 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.699453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43490 (local address 127.31.62.129:38073)
0925 10:59:12.699655 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:12.699662 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:12.711237 (+ 11575us) server_negotiation.cc:407] Connection header received
0925 10:59:12.711451 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:12.711458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:12.711589 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:12.711704 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:12.712239 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.713598 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.715470 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.716108 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.716214 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.718065 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:12.718080 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:12.718099 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:12.718867 (+ 768us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:12.718927 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:12.718952 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:12.719037 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:12.719322 (+ 285us) server_negotiation.cc:299] Negotiation successful
0925 10:59:12.719436 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 10:59:12.720530 17765 ts_tablet_manager.cc:1905] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 28beec99d7b84ef382620323518bf02f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:12.721383 17456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:12.722409 17764 ts_tablet_manager.cc:1905] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:12.729893 17764 ts_tablet_manager.cc:1918] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:12.730280 17764 log.cc:1196] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/b0282f5e5b9b4cd7a0a1a8f2ce1af0a0
I20260925 10:59:12.730902 17764 ts_tablet_manager.cc:1939] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P f2cc970f48b04b22a22949aa3190b89e: Deleting consensus metadata
I20260925 10:59:12.731894 17766 ts_tablet_manager.cc:1918] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:12.732108 17766 log.cc:1196] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/b0282f5e5b9b4cd7a0a1a8f2ce1af0a0
I20260925 10:59:12.732652 17766 ts_tablet_manager.cc:1939] T b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 P 127af10ca57a4378b91abea332b7fff1: Deleting consensus metadata
I20260925 10:59:12.733104 17259 catalog_manager.cc:4841] TS f2cc970f48b04b22a22949aa3190b89e (127.31.62.130:41023): tablet b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 (table test-table [id=b560e3a69fb04fe5878f5f407195d73f]) successfully deleted
I20260925 10:59:12.734391 17260 catalog_manager.cc:4841] TS 127af10ca57a4378b91abea332b7fff1 (127.31.62.131:34179): tablet b0282f5e5b9b4cd7a0a1a8f2ce1af0a0 (table test-table [id=b560e3a69fb04fe5878f5f407195d73f]) successfully deleted
I20260925 10:59:12.814324 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 17333
I20260925 10:59:12.829107 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 17465
W20260925 10:59:12.829245 17261 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.62.129:38073 (error 108)
I20260925 10:59:12.831404 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.830525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:38073 (local address 127.0.0.1:43492)
0925 10:59:12.831103 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:12.831231 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":358,"threads_started":1}
W20260925 10:59:12.831701 17261 catalog_manager.cc:4584] TS 28beec99d7b84ef382620323518bf02f (127.31.62.129:38073): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b0282f5e5b9b4cd7a0a1a8f2ce1af0a0: Network error: Client connection negotiation failed: client connection to 127.31.62.129:38073: connect: Connection refused (error 111)
I20260925 10:59:12.837621 17735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.830235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35488 (local address 127.31.62.130:41023)
0925 10:59:12.830425 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:12.830429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:12.830445 (+ 16us) server_negotiation.cc:407] Connection header received
0925 10:59:12.830498 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:12.830502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:12.830585 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:12.830676 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:12.831156 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.832256 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.833761 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.834435 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.834575 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.835779 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:12.835792 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:12.835813 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:12.836701 (+ 888us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:12.836772 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:12.836800 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:12.836964 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:12.837108 (+ 144us) server_negotiation.cc:299] Negotiation successful
0925 10:59:12.837230 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 10:59:12.838457 17587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:12.885437 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.884842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:38073 (local address 127.0.0.1:43504)
0925 10:59:12.885126 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:12.885243 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260925 10:59:12.886643 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.886195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:38073 (local address 127.0.0.1:43506)
0925 10:59:12.886386 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:12.886478 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260925 10:59:12.927228 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 17465
I20260925 10:59:12.941697 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 17595
I20260925 10:59:12.947120 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.946686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:38073 (local address 127.0.0.1:43514)
0925 10:59:12.946875 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:12.946955 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260925 10:59:12.948326 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.947911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:38073 (local address 127.0.0.1:43522)
0925 10:59:12.948076 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:12.948166 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260925 10:59:12.949282 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:12.942691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34380 (local address 127.31.62.131:34179)
0925 10:59:12.942975 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:12.942979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:12.942994 (+ 15us) server_negotiation.cc:407] Connection header received
0925 10:59:12.943053 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:12.943057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:12.943134 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:12.943231 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:12.943671 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.944453 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.945855 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:12.946345 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:12.946474 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:12.947681 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:12.947694 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:12.947712 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:12.948483 (+ 771us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:12.948548 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:12.948574 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:12.948680 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:12.948819 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 10:59:12.948938 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 10:59:12.950135 17717 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:13.029156 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:13.028587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:38073 (local address 127.0.0.1:43530)
0925 10:59:13.028823 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:13.029004 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260925 10:59:13.030256 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:13.029819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:38073 (local address 127.0.0.1:43534)
0925 10:59:13.030047 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:13.030126 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260925 10:59:13.070101 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 17595
I20260925 10:59:13.085664 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 17243
I20260925 10:59:13.086827 17304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:13.194986 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 17243
2026-09-25T10:59:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3033 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:13 dist-test-slave-nx7s krb5kdc[17776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:13 dist-test-slave-nx7s krb5kdc[17776](info): set up 2 sockets
Sep 25 10:59:13 dist-test-slave-nx7s krb5kdc[17776](info): commencing operation
krb5kdc: starting...
W20260925 10:59:14.225337 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:14 dist-test-slave-nx7s krb5kdc[17776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:59:14.469280 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:46287
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:39883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:46287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:14.793681 17789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:14.794027 17789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:14.794162 17789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:14.801620 17789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:14.801748 17789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:14.801832 17789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:14.801908 17789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:14.801965 17789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:14.811985 17789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:39883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:46287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:46287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:14.813402 17789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:14.814714 17789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:14.826139 17794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:14.826236 17795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:14.826256 17797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:14.827180 17789 server_base.cc:1034] running on GCE node
I20260925 10:59:14.827991 17789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:14.829473 17789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:14.830686 17789 hybrid_clock.cc:648] HybridClock initialized: now 1790333954830629 us; error 42 us; skew 500 ppm
I20260925 10:59:14.831043 17789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:14 dist-test-slave-nx7s krb5kdc[17776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333954, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:14.838200 17789 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:59:14.839955 17789 webserver.cc:460] Webserver started at http://127.31.62.190:44235/ using document root <none> and password file <none>
I20260925 10:59:14.840682 17789 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:14.840811 17789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:14.841144 17789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:14.844239 17789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "f3f3135c9e1642f09191bb529a9df4d4"
format_stamp: "Formatted at 2026-09-25 10:59:14 on dist-test-slave-nx7s"
server_key: "a84326a9e33c66a5e7144b3aea237164"
server_key_iv: "c122f0f6c06287f391cbca33701f38cb"
server_key_version: "encryptionkey@0"
I20260925 10:59:14.845103 17789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "f3f3135c9e1642f09191bb529a9df4d4"
format_stamp: "Formatted at 2026-09-25 10:59:14 on dist-test-slave-nx7s"
server_key: "a84326a9e33c66a5e7144b3aea237164"
server_key_iv: "c122f0f6c06287f391cbca33701f38cb"
server_key_version: "encryptionkey@0"
I20260925 10:59:14.850643 17789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:59:14.854389 17804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:14.855855 17789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:59:14.856040 17789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "f3f3135c9e1642f09191bb529a9df4d4"
format_stamp: "Formatted at 2026-09-25 10:59:14 on dist-test-slave-nx7s"
server_key: "a84326a9e33c66a5e7144b3aea237164"
server_key_iv: "c122f0f6c06287f391cbca33701f38cb"
server_key_version: "encryptionkey@0"
I20260925 10:59:14.856235 17789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:14.874017 17789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:14.878334 17789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:14.878608 17789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:14.896471 17789 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:46287
I20260925 10:59:14.896477 17855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:46287 every 8 connection(s)
I20260925 10:59:14.897853 17789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:14.901695 17856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:14.902047 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 17789
I20260925 10:59:14.902194 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:14.902664 31994 external_mini_cluster.cc:1428] Setting key 82690c83c9164c8fcd3e6110c0095b4e
I20260925 10:59:14.915700 17856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4: Bootstrap starting.
Sep 25 10:59:14 dist-test-slave-nx7s krb5kdc[17776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:59:14.918809 17856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:14.920130 17856 log.cc:824] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:14.923597 17856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4: No bootstrap required, opened a new log
I20260925 10:59:14.927766 17859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:14.904495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35828 (local address 127.31.62.190:46287)
0925 10:59:14.905288 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:14.905303 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:14.905351 (+ 48us) server_negotiation.cc:407] Connection header received
0925 10:59:14.908083 (+ 2732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:14.908113 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:14.908679 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:14.909170 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:14.910529 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:14.912480 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:14.914014 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:14.915026 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:14.915210 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:14.919102 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:14.919114 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:14.919141 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:14.919201 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:14.923008 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:14.923739 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:14.923748 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:14.923756 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:14.923967 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:14.924330 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:14.924334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:14.924337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:14.925151 (+ 814us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:14.925470 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:14.925873 (+ 403us) server_negotiation.cc:299] Negotiation successful
0925 10:59:14.926346 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":313,"threads_started":1}
I20260925 10:59:14.932806 17856 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f3135c9e1642f09191bb529a9df4d4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 46287 } }
I20260925 10:59:14.933265 17856 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:14.933369 17856 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3f3135c9e1642f09191bb529a9df4d4, State: Initialized, Role: FOLLOWER
I20260925 10:59:14.933961 17856 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f3135c9e1642f09191bb529a9df4d4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 46287 } }
I20260925 10:59:14.934183 17856 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:14.934306 17856 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:14.934468 17856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:14.936605 17856 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f3135c9e1642f09191bb529a9df4d4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 46287 } }
I20260925 10:59:14.937156 17856 leader_election.cc:302] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3f3135c9e1642f09191bb529a9df4d4; no voters:
I20260925 10:59:14.937742 17856 leader_election.cc:288] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:14.937871 17861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:14.938535 17861 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [term 1 LEADER]: Becoming Leader. State: Replica: f3f3135c9e1642f09191bb529a9df4d4, State: Running, Role: LEADER
I20260925 10:59:14.939091 17861 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f3135c9e1642f09191bb529a9df4d4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 46287 } }
I20260925 10:59:14.939798 17856 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:14.942044 17863 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3f3135c9e1642f09191bb529a9df4d4. Latest consensus state: current_term: 1 leader_uuid: "f3f3135c9e1642f09191bb529a9df4d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f3135c9e1642f09191bb529a9df4d4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 46287 } } }
I20260925 10:59:14.942056 17862 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3f3135c9e1642f09191bb529a9df4d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f3135c9e1642f09191bb529a9df4d4" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 46287 } } }
I20260925 10:59:14.942540 17862 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:14.942540 17863 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:14.943393 17867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:14.948731 17867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:14.962160 17867 catalog_manager.cc:1347] Generated new cluster ID: cfd125c7341c4273b9e1247812655edb
I20260925 10:59:14.962281 17867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:15.022871 17867 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:15.024821 17867 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:15.033998 17867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4: Generated new TSK 0
I20260925 10:59:15.035087 17867 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:59:15.090744 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:46287
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:39883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:15.424890 17882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:15.425194 17882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:15.425334 17882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:15.432821 17882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:15.432976 17882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:15.433127 17882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:59:15.443041 17882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:39883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:46287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:15.444341 17882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:15.445792 17882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:15.459079 17887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:15.459151 17888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:15.459075 17890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:15.460059 17882 server_base.cc:1034] running on GCE node
I20260925 10:59:15.460741 17882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:15.461830 17882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:15.463042 17882 hybrid_clock.cc:648] HybridClock initialized: now 1790333955462977 us; error 51 us; skew 500 ppm
I20260925 10:59:15.463390 17882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:15 dist-test-slave-nx7s krb5kdc[17776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333955, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:15.466563 17882 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:59:15.467954 17882 webserver.cc:460] Webserver started at http://127.31.62.129:33525/ using document root <none> and password file <none>
I20260925 10:59:15.468603 17882 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:15.468703 17882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:15.469008 17882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:15.471901 17882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "b7395f703227439ea58478d5ad7e1191"
format_stamp: "Formatted at 2026-09-25 10:59:15 on dist-test-slave-nx7s"
server_key: "679c4874839697fa6a4fe9cf19075705"
server_key_iv: "5da30694fab528a03de00129630cbd58"
server_key_version: "encryptionkey@0"
I20260925 10:59:15.472651 17882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "b7395f703227439ea58478d5ad7e1191"
format_stamp: "Formatted at 2026-09-25 10:59:15 on dist-test-slave-nx7s"
server_key: "679c4874839697fa6a4fe9cf19075705"
server_key_iv: "5da30694fab528a03de00129630cbd58"
server_key_version: "encryptionkey@0"
I20260925 10:59:15.478179 17882 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260925 10:59:15.481747 17897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:15.483212 17882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260925 10:59:15.483367 17882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "b7395f703227439ea58478d5ad7e1191"
format_stamp: "Formatted at 2026-09-25 10:59:15 on dist-test-slave-nx7s"
server_key: "679c4874839697fa6a4fe9cf19075705"
server_key_iv: "5da30694fab528a03de00129630cbd58"
server_key_version: "encryptionkey@0"
I20260925 10:59:15.483565 17882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:15.500523 17882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:15.504904 17882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:15.505177 17882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:15.506234 17882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:15.507562 17882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:15.507637 17882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:15.507758 17882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:15.507810 17882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:15.537299 17882 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:37335
I20260925 10:59:15.537339 18009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:37335 every 8 connection(s)
I20260925 10:59:15.538566 17882 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:59:15.547240 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 17882
I20260925 10:59:15.547394 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:59:15.547817 31994 external_mini_cluster.cc:1428] Setting key 4db6625ea9bcbdd04065c3e5332d7d2f
Sep 25 10:59:15 dist-test-slave-nx7s krb5kdc[17776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333955, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:59:15.558157 18012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:15.541041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:34165 (local address 127.31.62.190:46287)
0925 10:59:15.541498 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:15.541502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:15.542399 (+ 897us) server_negotiation.cc:407] Connection header received
0925 10:59:15.544195 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:15.544201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:15.544283 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:15.544382 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:15.546020 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:15.546747 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:15.548465 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:15.548962 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:15.549066 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:15.552679 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:15.552685 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:15.552688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:15.552723 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:15.555167 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:15.555884 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:15.555889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:15.555891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:15.556020 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:15.556436 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:15.556441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:15.556443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:15.556708 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:15.556837 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:15.557567 (+ 730us) server_negotiation.cc:299] Negotiation successful
0925 10:59:15.557794 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":253,"threads_started":1}
I20260925 10:59:15.559180 18013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:15.541490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:46287 (local address 127.31.62.129:34165)
0925 10:59:15.542239 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:15.542276 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:15.543931 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:15.544649 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:15.544658 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:15.545246 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:15.545853 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:15.545873 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:15.546932 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:15.546937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:15.548320 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:15.548333 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:15.549180 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:15.549186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:15.549461 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:15.550094 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:59:15.550116 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:15.552451 (+ 2335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:15.555305 (+ 2854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:15.555315 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:15.555333 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:15.555766 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:15.556124 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:15.556129 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:15.556133 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:15.556329 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:15.556830 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:15.556839 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:15.557215 (+ 376us) client_negotiation.cc:769] Sending connection context
0925 10:59:15.557471 (+ 256us) client_negotiation.cc:240] Negotiation successful
0925 10:59:15.558004 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":302,"threads_started":1}
I20260925 10:59:15.560725 18010 heartbeater.cc:347] Connected to a master server at 127.31.62.190:46287
I20260925 10:59:15.561089 18010 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:15.561975 18010 heartbeater.cc:511] Master 127.31.62.190:46287 requested a full tablet report, sending...
I20260925 10:59:15.564270 17821 ts_manager.cc:194] Registered new tserver with Master: b7395f703227439ea58478d5ad7e1191 (127.31.62.129:37335)
I20260925 10:59:15.565923 17821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:34165
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:59:15.600399 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:46287
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:39883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:15.923298 18016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:15.923576 18016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:15.923668 18016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:15.931064 18016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:15.931193 18016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:15.931319 18016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:59:15.941298 18016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:39883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:46287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:15.942592 18016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:15.943838 18016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:15.956350 18024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:15.956466 18021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:15.956459 18022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:15.957309 18016 server_base.cc:1034] running on GCE node
I20260925 10:59:15.957924 18016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:15.958853 18016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:15.960084 18016 hybrid_clock.cc:648] HybridClock initialized: now 1790333955960021 us; error 46 us; skew 500 ppm
I20260925 10:59:15.960377 18016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:15 dist-test-slave-nx7s krb5kdc[17776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333955, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:15.963507 18016 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:59:15.964941 18016 webserver.cc:460] Webserver started at http://127.31.62.130:34909/ using document root <none> and password file <none>
I20260925 10:59:15.965597 18016 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:15.965698 18016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:15.965950 18016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:15.968791 18016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "9f7629491803459589e4b1cd65ff6e14"
format_stamp: "Formatted at 2026-09-25 10:59:15 on dist-test-slave-nx7s"
server_key: "e13f32c6581a5cfe9aa6017617f3c3bd"
server_key_iv: "f1dc2c417b070e047ddb3769db44ff41"
server_key_version: "encryptionkey@0"
I20260925 10:59:15.969565 18016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "9f7629491803459589e4b1cd65ff6e14"
format_stamp: "Formatted at 2026-09-25 10:59:15 on dist-test-slave-nx7s"
server_key: "e13f32c6581a5cfe9aa6017617f3c3bd"
server_key_iv: "f1dc2c417b070e047ddb3769db44ff41"
server_key_version: "encryptionkey@0"
I20260925 10:59:15.974601 18016 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:59:15.978161 18031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:15.979508 18016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:59:15.979637 18016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "9f7629491803459589e4b1cd65ff6e14"
format_stamp: "Formatted at 2026-09-25 10:59:15 on dist-test-slave-nx7s"
server_key: "e13f32c6581a5cfe9aa6017617f3c3bd"
server_key_iv: "f1dc2c417b070e047ddb3769db44ff41"
server_key_version: "encryptionkey@0"
I20260925 10:59:15.979817 18016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:15.996670 18016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:16.001026 18016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:16.001269 18016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:16.002213 18016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:16.003618 18016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:16.003688 18016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:16.003780 18016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:16.003821 18016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:16.033668 18016 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:34939
I20260925 10:59:16.033713 18143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:34939 every 8 connection(s)
I20260925 10:59:16.035110 18016 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:59:16.043648 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 18016
I20260925 10:59:16.043824 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:59:16.044359 31994 external_mini_cluster.cc:1428] Setting key cb1518ec723076d4b08c2b5c3dd9e997
Sep 25 10:59:16 dist-test-slave-nx7s krb5kdc[17776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333955, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:59:16.056577 18012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:16.037736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:55843 (local address 127.31.62.190:46287)
0925 10:59:16.037932 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:16.037938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:16.039253 (+ 1315us) server_negotiation.cc:407] Connection header received
0925 10:59:16.041204 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:16.041211 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:16.041313 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:16.041433 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:16.043169 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.044247 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.046067 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.046693 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.046816 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:16.050570 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:16.050576 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:16.050579 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:16.050615 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:16.053342 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:16.054249 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:16.054254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:16.054257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:16.054421 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:16.054817 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:16.054821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:16.054824 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:16.055150 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:16.055288 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:16.056000 (+ 712us) server_negotiation.cc:299] Negotiation successful
0925 10:59:16.056236 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 10:59:16.057484 18146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:16.038356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:46287 (local address 127.31.62.130:55843)
0925 10:59:16.039089 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:16.039132 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:16.040917 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:16.041704 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:16.041717 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:16.042381 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:16.042992 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:16.043011 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.044425 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.044431 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:16.045939 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:16.045956 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.046914 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.046920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:16.047155 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:16.047791 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:59:16.047820 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:16.050209 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:16.053497 (+ 3288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:16.053507 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:16.053527 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:16.054105 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:16.054519 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:16.054523 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:16.054525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:16.054702 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:16.055267 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:16.055274 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:16.055656 (+ 382us) client_negotiation.cc:769] Sending connection context
0925 10:59:16.055894 (+ 238us) client_negotiation.cc:240] Negotiation successful
0925 10:59:16.056306 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":255,"threads_started":1}
I20260925 10:59:16.058923 18144 heartbeater.cc:347] Connected to a master server at 127.31.62.190:46287
I20260925 10:59:16.059242 18144 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:16.060140 18144 heartbeater.cc:511] Master 127.31.62.190:46287 requested a full tablet report, sending...
I20260925 10:59:16.061769 17821 ts_manager.cc:194] Registered new tserver with Master: 9f7629491803459589e4b1cd65ff6e14 (127.31.62.130:34939)
I20260925 10:59:16.062599 17821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:55843
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:59:16.101152 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:46287
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:39883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:16.428934 18149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:16.429242 18149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:16.429369 18149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:16.437300 18149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:16.437427 18149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:16.437585 18149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:59:16.447758 18149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:39883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:46287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:16.449110 18149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:16.450467 18149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:16.463392 18155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:16.463471 18154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:16.463435 18157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:16.464293 18149 server_base.cc:1034] running on GCE node
I20260925 10:59:16.464957 18149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:16.465917 18149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:16.467127 18149 hybrid_clock.cc:648] HybridClock initialized: now 1790333956467065 us; error 43 us; skew 500 ppm
I20260925 10:59:16.467442 18149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:16 dist-test-slave-nx7s krb5kdc[17776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333956, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:16.470471 18149 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:59:16.471932 18149 webserver.cc:460] Webserver started at http://127.31.62.131:38769/ using document root <none> and password file <none>
I20260925 10:59:16.472594 18149 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:16.472694 18149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:16.473028 18149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:16.476011 18149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "98a544033bfc4f9fbc2fb8adcc4104f1"
format_stamp: "Formatted at 2026-09-25 10:59:16 on dist-test-slave-nx7s"
server_key: "d9cc4f970ccb1694dcc7d3ffd24e601f"
server_key_iv: "f3dcd782e2d4bf5e7cbca38dbb58bf05"
server_key_version: "encryptionkey@0"
I20260925 10:59:16.476764 18149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "98a544033bfc4f9fbc2fb8adcc4104f1"
format_stamp: "Formatted at 2026-09-25 10:59:16 on dist-test-slave-nx7s"
server_key: "d9cc4f970ccb1694dcc7d3ffd24e601f"
server_key_iv: "f3dcd782e2d4bf5e7cbca38dbb58bf05"
server_key_version: "encryptionkey@0"
I20260925 10:59:16.482599 18149 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:59:16.486515 18164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:16.488022 18149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:59:16.488185 18149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "98a544033bfc4f9fbc2fb8adcc4104f1"
format_stamp: "Formatted at 2026-09-25 10:59:16 on dist-test-slave-nx7s"
server_key: "d9cc4f970ccb1694dcc7d3ffd24e601f"
server_key_iv: "f3dcd782e2d4bf5e7cbca38dbb58bf05"
server_key_version: "encryptionkey@0"
I20260925 10:59:16.488373 18149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:16.505245 18149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:16.509397 18149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:16.509666 18149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:16.510622 18149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:16.511968 18149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:16.512044 18149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:16.512161 18149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:16.512214 18149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:16.544266 18149 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:40649
I20260925 10:59:16.544294 18276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:40649 every 8 connection(s)
I20260925 10:59:16.545782 18149 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:59:16.554194 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 18149
I20260925 10:59:16.554380 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:59:16.554853 31994 external_mini_cluster.cc:1428] Setting key f3e665bd26e13cbef6edf9d5f8644a35
Sep 25 10:59:16 dist-test-slave-nx7s krb5kdc[17776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333956, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:59:16.565637 18012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:16.548381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:48947 (local address 127.31.62.190:46287)
0925 10:59:16.548565 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:16.548570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:16.549756 (+ 1186us) server_negotiation.cc:407] Connection header received
0925 10:59:16.551587 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:16.551593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:16.551711 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:16.551823 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:16.553409 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.554369 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.556102 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.556606 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.556725 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:16.560138 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:16.560143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:16.560146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:16.560187 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:16.562695 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:16.563459 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:16.563463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:16.563465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:16.563603 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:16.564018 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:16.564022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:16.564024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:16.564297 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:16.564405 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:16.565151 (+ 746us) server_negotiation.cc:299] Negotiation successful
0925 10:59:16.565327 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 10:59:16.566526 18279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:16.548821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:46287 (local address 127.31.62.131:48947)
0925 10:59:16.549583 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:16.549629 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:16.551304 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:16.552071 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:16.552079 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:16.552599 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:16.553249 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:16.553267 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.554522 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.554526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:16.555957 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:16.555973 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.556800 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.556805 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:16.557100 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:16.557755 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:59:16.557778 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:16.559935 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:16.562827 (+ 2892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:16.562834 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:16.562848 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:16.563294 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:16.563703 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:16.563706 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:16.563708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:16.563911 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:16.564413 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:16.564419 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:16.564762 (+ 343us) client_negotiation.cc:769] Sending connection context
0925 10:59:16.565038 (+ 276us) client_negotiation.cc:240] Negotiation successful
0925 10:59:16.565451 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":288,"threads_started":1}
I20260925 10:59:16.567898 18277 heartbeater.cc:347] Connected to a master server at 127.31.62.190:46287
I20260925 10:59:16.568197 18277 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:16.568997 18277 heartbeater.cc:511] Master 127.31.62.190:46287 requested a full tablet report, sending...
I20260925 10:59:16.569406 18010 heartbeater.cc:503] Master 127.31.62.190:46287 was elected leader, sending a full tablet report...
I20260925 10:59:16.570559 17820 ts_manager.cc:194] Registered new tserver with Master: 98a544033bfc4f9fbc2fb8adcc4104f1 (127.31.62.131:40649)
I20260925 10:59:16.571437 17820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:48947
I20260925 10:59:16.583163 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:59:16.600229 18012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:16.587369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40262 (local address 127.31.62.190:46287)
0925 10:59:16.587588 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:16.587594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:16.587989 (+ 395us) server_negotiation.cc:407] Connection header received
0925 10:59:16.588176 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:16.588181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:16.588248 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:16.588329 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:16.589736 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.590469 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.591804 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.592266 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.592372 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:16.595041 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:16.595047 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:16.595050 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:16.595088 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:16.597617 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:16.598346 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:16.598349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:16.598352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:16.598474 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:16.598858 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:16.598861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:16.598863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:16.599164 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:16.599387 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:16.599854 (+ 467us) server_negotiation.cc:299] Negotiation successful
0925 10:59:16.599991 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260925 10:59:16.604254 17820 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40262:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:59:16.611349 17820 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:59:16.645505 18291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:16.624342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:40649 (local address 127.0.0.1:59954)
0925 10:59:16.635853 (+ 11511us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:16.635883 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:16.636080 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:16.638153 (+ 2073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:16.638158 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:16.638189 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:16.638705 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:16.638717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.640715 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.640719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:16.643151 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:16.643160 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.644438 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.644444 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:16.644646 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:16.644700 (+ 54us) client_negotiation.cc:769] Sending connection context
0925 10:59:16.644809 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 10:59:16.645023 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11300,"mutex_wait_us":87,"spinlock_wait_cycles":33408,"thread_start_us":11884,"threads_started":1}
I20260925 10:59:16.645889 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:16.624710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59954 (local address 127.31.62.131:40649)
0925 10:59:16.637462 (+ 12752us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:16.637473 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:16.637510 (+ 37us) server_negotiation.cc:407] Connection header received
0925 10:59:16.637615 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:16.637623 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:16.637900 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:16.638913 (+ 1013us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:16.639188 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.640579 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.643310 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.644248 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.644367 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:16.644625 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:16.644896 (+ 271us) server_negotiation.cc:299] Negotiation successful
0925 10:59:16.645542 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12556,"thread_start_us":12386,"threads_started":1}
I20260925 10:59:16.645951 18289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:16.623947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:37335 (local address 127.0.0.1:60806)
0925 10:59:16.624668 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:16.624710 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:16.624933 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:16.637930 (+ 12997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:16.637939 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:16.637985 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:16.638574 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:16.638587 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.640495 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.640500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:16.643136 (+ 2636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:16.643154 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.645090 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.645096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:16.645267 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:16.645307 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 10:59:16.645406 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 10:59:16.645538 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":19,"thread_start_us":565,"threads_started":1}
I20260925 10:59:16.646075 18293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:16.625432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:34939 (local address 127.0.0.1:34446)
0925 10:59:16.638199 (+ 12767us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:16.638218 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:16.638380 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:16.639278 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:16.639282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:16.639306 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:16.639658 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:16.639670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.641368 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.641373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:16.643251 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:16.643265 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.645321 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.645328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:16.645506 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:16.645534 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 10:59:16.645603 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 10:59:16.645716 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12634,"mutex_wait_us":21,"spinlock_wait_cycles":768,"thread_start_us":12464,"threads_started":1}
I20260925 10:59:16.646972 18290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:16.624205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60806 (local address 127.31.62.129:37335)
0925 10:59:16.637064 (+ 12859us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:16.637075 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:16.637120 (+ 45us) server_negotiation.cc:407] Connection header received
0925 10:59:16.637236 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:16.637245 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:16.637571 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:16.638782 (+ 1211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:16.639048 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.640375 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.643359 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.644832 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.646187 (+ 1355us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:16.646429 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:16.646496 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 10:59:16.646661 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12618,"thread_start_us":12426,"threads_started":1}
I20260925 10:59:16.647008 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:16.625893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34446 (local address 127.31.62.130:34939)
0925 10:59:16.637727 (+ 11834us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:16.637736 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:16.638521 (+ 785us) server_negotiation.cc:407] Connection header received
0925 10:59:16.638623 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:16.638631 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:16.638883 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:16.639065 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:16.639842 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.641199 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.643386 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.644884 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.645000 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:16.645186 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:16.646572 (+ 1386us) server_negotiation.cc:299] Negotiation successful
0925 10:59:16.646727 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11682,"thread_start_us":11583,"threads_started":1}
I20260925 10:59:16.649811 18212 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa2877e6ccf467cbb35489ab4c2959f (DEFAULT_TABLE table=test-table [id=323e818e48f74ab183786dcefdaebf9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:16.650354 17945 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa2877e6ccf467cbb35489ab4c2959f (DEFAULT_TABLE table=test-table [id=323e818e48f74ab183786dcefdaebf9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:16.651144 18212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa2877e6ccf467cbb35489ab4c2959f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:16.651093 18079 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa2877e6ccf467cbb35489ab4c2959f (DEFAULT_TABLE table=test-table [id=323e818e48f74ab183786dcefdaebf9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:16.651940 17945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa2877e6ccf467cbb35489ab4c2959f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:16.652415 18079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa2877e6ccf467cbb35489ab4c2959f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:16.664192 18295 tablet_bootstrap.cc:492] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1: Bootstrap starting.
I20260925 10:59:16.667297 18295 tablet_bootstrap.cc:654] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:16.668272 18295 log.cc:824] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:16.669533 18297 tablet_bootstrap.cc:492] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191: Bootstrap starting.
I20260925 10:59:16.670018 18296 tablet_bootstrap.cc:492] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14: Bootstrap starting.
I20260925 10:59:16.673221 18297 tablet_bootstrap.cc:654] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:16.673547 18296 tablet_bootstrap.cc:654] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:16.674348 18297 log.cc:824] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:16.674698 18296 log.cc:824] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:16.675839 18295 tablet_bootstrap.cc:492] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1: No bootstrap required, opened a new log
I20260925 10:59:16.676164 18295 ts_tablet_manager.cc:1397] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260925 10:59:16.676951 18297 tablet_bootstrap.cc:492] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191: No bootstrap required, opened a new log
I20260925 10:59:16.677314 18297 ts_tablet_manager.cc:1397] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260925 10:59:16.679567 18296 tablet_bootstrap.cc:492] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14: No bootstrap required, opened a new log
I20260925 10:59:16.679906 18296 ts_tablet_manager.cc:1397] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.006s
I20260925 10:59:16.685425 18297 raft_consensus.cc:348] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7395f703227439ea58478d5ad7e1191" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37335 } } peers { permanent_uuid: "9f7629491803459589e4b1cd65ff6e14" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34939 } } peers { permanent_uuid: "98a544033bfc4f9fbc2fb8adcc4104f1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40649 } }
I20260925 10:59:16.685881 18297 raft_consensus.cc:374] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:16.686003 18297 raft_consensus.cc:729] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7395f703227439ea58478d5ad7e1191, State: Initialized, Role: FOLLOWER
I20260925 10:59:16.686942 18297 consensus_queue.cc:260] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7395f703227439ea58478d5ad7e1191" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37335 } } peers { permanent_uuid: "9f7629491803459589e4b1cd65ff6e14" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34939 } } peers { permanent_uuid: "98a544033bfc4f9fbc2fb8adcc4104f1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40649 } }
I20260925 10:59:16.686954 18295 raft_consensus.cc:348] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7395f703227439ea58478d5ad7e1191" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37335 } } peers { permanent_uuid: "9f7629491803459589e4b1cd65ff6e14" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34939 } } peers { permanent_uuid: "98a544033bfc4f9fbc2fb8adcc4104f1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40649 } }
I20260925 10:59:16.687457 18295 raft_consensus.cc:374] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:16.687603 18295 raft_consensus.cc:729] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98a544033bfc4f9fbc2fb8adcc4104f1, State: Initialized, Role: FOLLOWER
I20260925 10:59:16.688397 18295 consensus_queue.cc:260] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7395f703227439ea58478d5ad7e1191" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37335 } } peers { permanent_uuid: "9f7629491803459589e4b1cd65ff6e14" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34939 } } peers { permanent_uuid: "98a544033bfc4f9fbc2fb8adcc4104f1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40649 } }
I20260925 10:59:16.688822 18297 ts_tablet_manager.cc:1428] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260925 10:59:16.689504 18277 heartbeater.cc:503] Master 127.31.62.190:46287 was elected leader, sending a full tablet report...
I20260925 10:59:16.690156 18295 ts_tablet_manager.cc:1428] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
I20260925 10:59:16.691619 18296 raft_consensus.cc:348] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7395f703227439ea58478d5ad7e1191" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37335 } } peers { permanent_uuid: "9f7629491803459589e4b1cd65ff6e14" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34939 } } peers { permanent_uuid: "98a544033bfc4f9fbc2fb8adcc4104f1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40649 } }
I20260925 10:59:16.692114 18296 raft_consensus.cc:374] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:16.692214 18296 raft_consensus.cc:729] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f7629491803459589e4b1cd65ff6e14, State: Initialized, Role: FOLLOWER
I20260925 10:59:16.692828 18296 consensus_queue.cc:260] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7395f703227439ea58478d5ad7e1191" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37335 } } peers { permanent_uuid: "9f7629491803459589e4b1cd65ff6e14" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34939 } } peers { permanent_uuid: "98a544033bfc4f9fbc2fb8adcc4104f1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40649 } }
I20260925 10:59:16.694015 18144 heartbeater.cc:503] Master 127.31.62.190:46287 was elected leader, sending a full tablet report...
I20260925 10:59:16.694584 18296 ts_tablet_manager.cc:1428] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14: Time spent starting tablet: real 0.015s user 0.008s sys 0.004s
W20260925 10:59:16.787127 18145 tablet.cc:2369] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:59:16.790908 18011 tablet.cc:2369] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:59:16.797423 18278 tablet.cc:2369] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:59:16.987303 18302 raft_consensus.cc:482] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:59:16.987653 18302 raft_consensus.cc:504] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7395f703227439ea58478d5ad7e1191" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37335 } } peers { permanent_uuid: "9f7629491803459589e4b1cd65ff6e14" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34939 } } peers { permanent_uuid: "98a544033bfc4f9fbc2fb8adcc4104f1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40649 } }
I20260925 10:59:16.989457 18302 leader_election.cc:288] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7395f703227439ea58478d5ad7e1191 (127.31.62.129:37335), 9f7629491803459589e4b1cd65ff6e14 (127.31.62.130:34939)
I20260925 10:59:16.995062 18279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:16.989739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:37335 (local address 127.31.62.131:43319)
0925 10:59:16.989913 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:16.989970 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:16.990094 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:16.990620 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:16.990623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:16.990648 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:16.990792 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:16.990799 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.991778 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.991783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:16.993255 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:16.993266 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.994354 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.994360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:16.994537 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:16.994561 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 10:59:16.994639 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 10:59:16.994769 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":18}
I20260925 10:59:16.995044 18290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:16.989812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:43319 (local address 127.31.62.129:37335)
0925 10:59:16.990011 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:16.990016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:16.990035 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:59:16.990279 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:16.990287 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:16.990374 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:16.990465 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:16.990941 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.991669 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.993442 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.994218 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.994357 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:16.994430 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:16.994681 (+ 251us) server_negotiation.cc:299] Negotiation successful
0925 10:59:16.994800 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 10:59:16.995684 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:16.990385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:35105 (local address 127.31.62.130:34939)
0925 10:59:16.990571 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:16.990577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:16.990741 (+ 164us) server_negotiation.cc:407] Connection header received
0925 10:59:16.990921 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:16.990927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:16.990987 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:16.991057 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:16.991518 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.992211 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.993984 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.994852 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.995011 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:16.995079 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:16.995380 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 10:59:16.995504 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 10:59:16.995824 18304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:16.990129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:34939 (local address 127.31.62.131:35105)
0925 10:59:16.990644 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:16.990660 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:16.990795 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:16.991080 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:16.991083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:16.991105 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:16.991387 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:16.991397 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.992351 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.992356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:16.993767 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:16.993778 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:16.995061 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:16.995067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:16.995222 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:16.995251 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 10:59:16.995345 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 10:59:16.995476 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":17,"thread_start_us":243,"threads_started":1}
I20260925 10:59:16.996264 17965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa2877e6ccf467cbb35489ab4c2959f" candidate_uuid: "98a544033bfc4f9fbc2fb8adcc4104f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7395f703227439ea58478d5ad7e1191" is_pre_election: true
I20260925 10:59:16.996759 17965 raft_consensus.cc:2454] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98a544033bfc4f9fbc2fb8adcc4104f1 in term 0.
I20260925 10:59:16.997542 18166 leader_election.cc:302] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98a544033bfc4f9fbc2fb8adcc4104f1, b7395f703227439ea58478d5ad7e1191; no voters:
I20260925 10:59:16.997691 18099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa2877e6ccf467cbb35489ab4c2959f" candidate_uuid: "98a544033bfc4f9fbc2fb8adcc4104f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f7629491803459589e4b1cd65ff6e14" is_pre_election: true
I20260925 10:59:16.997947 18302 raft_consensus.cc:2781] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:59:16.998099 18302 raft_consensus.cc:482] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:59:16.998202 18302 raft_consensus.cc:3037] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:16.998224 18099 raft_consensus.cc:2454] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98a544033bfc4f9fbc2fb8adcc4104f1 in term 0.
I20260925 10:59:17.000443 18302 raft_consensus.cc:504] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7395f703227439ea58478d5ad7e1191" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37335 } } peers { permanent_uuid: "9f7629491803459589e4b1cd65ff6e14" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34939 } } peers { permanent_uuid: "98a544033bfc4f9fbc2fb8adcc4104f1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40649 } }
I20260925 10:59:17.001024 18302 leader_election.cc:288] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [CANDIDATE]: Term 1 election: Requested vote from peers b7395f703227439ea58478d5ad7e1191 (127.31.62.129:37335), 9f7629491803459589e4b1cd65ff6e14 (127.31.62.130:34939)
I20260925 10:59:17.001533 17965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa2877e6ccf467cbb35489ab4c2959f" candidate_uuid: "98a544033bfc4f9fbc2fb8adcc4104f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7395f703227439ea58478d5ad7e1191"
I20260925 10:59:17.001577 18099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa2877e6ccf467cbb35489ab4c2959f" candidate_uuid: "98a544033bfc4f9fbc2fb8adcc4104f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f7629491803459589e4b1cd65ff6e14"
I20260925 10:59:17.001768 17965 raft_consensus.cc:3037] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:17.001896 18099 raft_consensus.cc:3037] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:17.005080 18099 raft_consensus.cc:2454] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98a544033bfc4f9fbc2fb8adcc4104f1 in term 1.
I20260925 10:59:17.005080 17965 raft_consensus.cc:2454] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98a544033bfc4f9fbc2fb8adcc4104f1 in term 1.
I20260925 10:59:17.005566 18166 leader_election.cc:302] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98a544033bfc4f9fbc2fb8adcc4104f1, b7395f703227439ea58478d5ad7e1191; no voters:
I20260925 10:59:17.005888 18302 raft_consensus.cc:2781] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:17.006341 18302 raft_consensus.cc:686] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [term 1 LEADER]: Becoming Leader. State: Replica: 98a544033bfc4f9fbc2fb8adcc4104f1, State: Running, Role: LEADER
I20260925 10:59:17.006983 18302 consensus_queue.cc:237] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7395f703227439ea58478d5ad7e1191" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37335 } } peers { permanent_uuid: "9f7629491803459589e4b1cd65ff6e14" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34939 } } peers { permanent_uuid: "98a544033bfc4f9fbc2fb8adcc4104f1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40649 } }
I20260925 10:59:17.012017 17821 catalog_manager.cc:5495] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 98a544033bfc4f9fbc2fb8adcc4104f1 (127.31.62.131). New cstate: current_term: 1 leader_uuid: "98a544033bfc4f9fbc2fb8adcc4104f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7395f703227439ea58478d5ad7e1191" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f7629491803459589e4b1cd65ff6e14" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98a544033bfc4f9fbc2fb8adcc4104f1" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 40649 } health_report { overall_health: HEALTHY } } }
I20260925 10:59:17.109822 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:17.103891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59958 (local address 127.31.62.131:40649)
0925 10:59:17.104102 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:17.104107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:17.104323 (+ 216us) server_negotiation.cc:407] Connection header received
0925 10:59:17.104544 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:17.104549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:17.104636 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:17.104772 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:59:17.105259 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:17.106097 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:17.107554 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:17.107964 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:17.108054 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:17.108553 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:59:17.109167 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:59:17.109305 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:17.109509 (+ 204us) server_negotiation.cc:299] Negotiation successful
0925 10:59:17.109561 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 10:59:17.118069 17965 raft_consensus.cc:1261] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191 [term 1 FOLLOWER]: Refusing update from remote peer 98a544033bfc4f9fbc2fb8adcc4104f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:59:17.118069 18099 raft_consensus.cc:1261] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14 [term 1 FOLLOWER]: Refusing update from remote peer 98a544033bfc4f9fbc2fb8adcc4104f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:59:17.119089 18305 consensus_queue.cc:1035] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f7629491803459589e4b1cd65ff6e14" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 34939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:59:17.119473 18302 consensus_queue.cc:1035] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7395f703227439ea58478d5ad7e1191" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 37335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:59:17.146425 17820 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:59:17.146747 17820 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:59:17.150017 17820 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3f3135c9e1642f09191bb529a9df4d4: Sending DeleteTablet for 3 replicas of tablet 1aa2877e6ccf467cbb35489ab4c2959f
I20260925 10:59:17.151183 17945 tablet_service.cc:1514] Processing DeleteTablet for tablet 1aa2877e6ccf467cbb35489ab4c2959f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:17 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:60806
I20260925 10:59:17.151187 18212 tablet_service.cc:1514] Processing DeleteTablet for tablet 1aa2877e6ccf467cbb35489ab4c2959f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:17 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:59954
I20260925 10:59:17.151943 18079 tablet_service.cc:1514] Processing DeleteTablet for tablet 1aa2877e6ccf467cbb35489ab4c2959f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:17 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:34446
I20260925 10:59:17.152496 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 17882
I20260925 10:59:17.163463 18321 tablet_replica.cc:330] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14: stopping tablet replica
I20260925 10:59:17.163677 18320 tablet_replica.cc:330] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1: stopping tablet replica
I20260925 10:59:17.163902 18321 raft_consensus.cc:2229] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:59:17.164117 18320 raft_consensus.cc:2229] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:59:17.164372 18321 raft_consensus.cc:2258] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:17.164877 18320 raft_consensus.cc:2258] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:17.165639 18319 tablet_replica.cc:330] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191: stopping tablet replica
I20260925 10:59:17.166093 18319 raft_consensus.cc:2229] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:59:17.166581 18319 raft_consensus.cc:2258] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:17.167387 18321 ts_tablet_manager.cc:1905] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:17.169983 18320 ts_tablet_manager.cc:1905] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:17.171360 18319 ts_tablet_manager.cc:1905] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:17.174559 18321 ts_tablet_manager.cc:1918] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:17.174795 18321 log.cc:1196] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/1aa2877e6ccf467cbb35489ab4c2959f
I20260925 10:59:17.175386 18321 ts_tablet_manager.cc:1939] T 1aa2877e6ccf467cbb35489ab4c2959f P 9f7629491803459589e4b1cd65ff6e14: Deleting consensus metadata
I20260925 10:59:17.177304 18320 ts_tablet_manager.cc:1918] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:17.177482 17806 catalog_manager.cc:4841] TS 9f7629491803459589e4b1cd65ff6e14 (127.31.62.130:34939): tablet 1aa2877e6ccf467cbb35489ab4c2959f (table test-table [id=323e818e48f74ab183786dcefdaebf9d]) successfully deleted
I20260925 10:59:17.177587 18320 log.cc:1196] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/1aa2877e6ccf467cbb35489ab4c2959f
I20260925 10:59:17.178112 18320 ts_tablet_manager.cc:1939] T 1aa2877e6ccf467cbb35489ab4c2959f P 98a544033bfc4f9fbc2fb8adcc4104f1: Deleting consensus metadata
I20260925 10:59:17.179612 18319 ts_tablet_manager.cc:1918] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:17.179816 18319 log.cc:1196] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/1aa2877e6ccf467cbb35489ab4c2959f
I20260925 10:59:17.180006 17807 catalog_manager.cc:4841] TS 98a544033bfc4f9fbc2fb8adcc4104f1 (127.31.62.131:40649): tablet 1aa2877e6ccf467cbb35489ab4c2959f (table test-table [id=323e818e48f74ab183786dcefdaebf9d]) successfully deleted
I20260925 10:59:17.180369 18319 ts_tablet_manager.cc:1939] T 1aa2877e6ccf467cbb35489ab4c2959f P b7395f703227439ea58478d5ad7e1191: Deleting consensus metadata
I20260925 10:59:17.181916 17806 catalog_manager.cc:4841] TS b7395f703227439ea58478d5ad7e1191 (127.31.62.129:37335): tablet 1aa2877e6ccf467cbb35489ab4c2959f (table test-table [id=323e818e48f74ab183786dcefdaebf9d]) successfully deleted
Sep 25 10:59:17 dist-test-slave-nx7s krb5kdc[17776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:59:17.192257 18290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:17.154075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60810 (local address 127.31.62.129:37335)
0925 10:59:17.154268 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:17.154276 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:17.161863 (+ 7587us) server_negotiation.cc:407] Connection header received
0925 10:59:17.162089 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:17.162096 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:17.162204 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:17.162309 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:17.168738 (+ 6429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:17.169909 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:17.178973 (+ 9064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:17.179662 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:17.179775 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:17.185273 (+ 5498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:17.185293 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:17.185314 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:17.185362 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:17.189136 (+ 3774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:17.190019 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:17.190028 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:17.190032 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:17.190197 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:17.190610 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:17.190615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:17.190619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:17.191075 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:17.191381 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:17.191634 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 10:59:17.191856 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 10:59:17.192986 18005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:17.299728 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 17882
I20260925 10:59:17.321668 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 18016
Sep 25 10:59:17 dist-test-slave-nx7s krb5kdc[17776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:59:17.338763 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:17.322844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34456 (local address 127.31.62.130:34939)
0925 10:59:17.323014 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:17.323019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:17.323037 (+ 18us) server_negotiation.cc:407] Connection header received
0925 10:59:17.323100 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:17.323104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:17.323176 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:17.323267 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:17.324871 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:17.325925 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:17.327770 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:17.328302 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:17.328431 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:17.333094 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:17.333109 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:17.333125 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:17.333169 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:17.335887 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:17.336601 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:17.336617 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:17.336619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:17.336760 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:17.337180 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:17.337184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:17.337186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:17.337577 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:17.337880 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:17.338190 (+ 310us) server_negotiation.cc:299] Negotiation successful
0925 10:59:17.338405 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 10:59:17.339511 18139 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:17.477763 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 18016
I20260925 10:59:17.491986 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 18149
Sep 25 10:59:17 dist-test-slave-nx7s krb5kdc[17776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:59:17.509196 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:17.492935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59960 (local address 127.31.62.131:40649)
0925 10:59:17.493143 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:17.493149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:17.493178 (+ 29us) server_negotiation.cc:407] Connection header received
0925 10:59:17.493389 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:17.493393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:17.493474 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:17.493566 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:17.494885 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:17.495739 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:17.497207 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:17.497677 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:17.497806 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:17.502641 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:17.502659 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:17.502674 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:17.502722 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:17.506143 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:17.507031 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:17.507039 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:17.507041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:17.507217 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:17.507600 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:17.507604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:17.507606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:17.507969 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:17.508275 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:17.508583 (+ 308us) server_negotiation.cc:299] Negotiation successful
0925 10:59:17.508796 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 10:59:17.510082 18272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:17.633162 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 18149
I20260925 10:59:17.649569 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 17789
I20260925 10:59:17.650617 17851 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:17.760594 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 17789
2026-09-25T10:59:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4566 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-25T10:59:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:17Z Disabled control of system clock
I20260925 10:59:17.832221 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:37177
--webserver_interface=127.31.62.190
--webserver_port=0
--builtin_ntp_servers=127.31.62.148:44859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:37177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 10:59:18.165469 18330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:18.165788 18330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:18.165912 18330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:18.174476 18330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:18.174603 18330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:18.174649 18330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:18.174731 18330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:18.174801 18330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:18.184711 18330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:37177
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:37177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:18.186098 18330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:18.187956 18330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:18.199920 18335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:18.199949 18336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:18.199920 18338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:18.200619 18330 server_base.cc:1034] running on GCE node
I20260925 10:59:18.201508 18330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:18.203152 18330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:18.204397 18330 hybrid_clock.cc:648] HybridClock initialized: now 1790333958204320 us; error 60 us; skew 500 ppm
I20260925 10:59:18.204810 18330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:59:18.207479 18330 webserver.cc:460] Webserver started at http://127.31.62.190:43039/ using document root <none> and password file <none>
I20260925 10:59:18.208343 18330 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:18.208555 18330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:18.209028 18330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:18.212230 18330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "faa82b1e5f6f4f54b06b6d1b91c553ab"
format_stamp: "Formatted at 2026-09-25 10:59:18 on dist-test-slave-nx7s"
server_key: "2b3dc0000c282dcc2c22fd374b3f7d2e"
server_key_iv: "a2e58dd3f426e73c82a86ecbc6e10f4b"
server_key_version: "encryptionkey@0"
I20260925 10:59:18.213160 18330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "faa82b1e5f6f4f54b06b6d1b91c553ab"
format_stamp: "Formatted at 2026-09-25 10:59:18 on dist-test-slave-nx7s"
server_key: "2b3dc0000c282dcc2c22fd374b3f7d2e"
server_key_iv: "a2e58dd3f426e73c82a86ecbc6e10f4b"
server_key_version: "encryptionkey@0"
I20260925 10:59:18.218713 18330 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 10:59:18.222636 18344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:18.224141 18330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:59:18.224323 18330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "faa82b1e5f6f4f54b06b6d1b91c553ab"
format_stamp: "Formatted at 2026-09-25 10:59:18 on dist-test-slave-nx7s"
server_key: "2b3dc0000c282dcc2c22fd374b3f7d2e"
server_key_iv: "a2e58dd3f426e73c82a86ecbc6e10f4b"
server_key_version: "encryptionkey@0"
I20260925 10:59:18.224557 18330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:18.247501 18330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:18.248382 18330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:18.248658 18330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:18.266562 18330 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:37177
I20260925 10:59:18.266538 18395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:37177 every 8 connection(s)
I20260925 10:59:18.267998 18330 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:18.272163 18396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:18.277012 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 18330
I20260925 10:59:18.277186 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:18.277604 31994 external_mini_cluster.cc:1428] Setting key 0117ea2a260207e60608d71d61155704
I20260925 10:59:18.284644 18396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab: Bootstrap starting.
I20260925 10:59:18.288020 18396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:18.289204 18396 log.cc:824] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:18.292065 18396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab: No bootstrap required, opened a new log
I20260925 10:59:18.296437 18399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:18.279457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50150 (local address 127.31.62.190:37177)
0925 10:59:18.285443 (+ 5986us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:18.285456 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:18.285505 (+ 49us) server_negotiation.cc:407] Connection header received
0925 10:59:18.287329 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:18.287360 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:18.287890 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:18.288306 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:18.288944 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:18.290295 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:18.291742 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:18.292479 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:18.292666 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:18.293838 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:18.293846 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:18.293865 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:18.294511 (+ 646us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:18.294607 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:18.294630 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:18.294752 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:18.294992 (+ 240us) server_negotiation.cc:299] Negotiation successful
0925 10:59:18.295333 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5499,"thread_start_us":1744,"threads_started":1}
I20260925 10:59:18.300745 18396 raft_consensus.cc:348] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa82b1e5f6f4f54b06b6d1b91c553ab" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37177 } }
I20260925 10:59:18.301177 18396 raft_consensus.cc:374] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:18.301268 18396 raft_consensus.cc:729] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faa82b1e5f6f4f54b06b6d1b91c553ab, State: Initialized, Role: FOLLOWER
I20260925 10:59:18.301795 18396 consensus_queue.cc:260] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa82b1e5f6f4f54b06b6d1b91c553ab" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37177 } }
I20260925 10:59:18.302011 18396 raft_consensus.cc:388] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:18.302127 18396 raft_consensus.cc:482] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:18.302277 18396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:18.304526 18396 raft_consensus.cc:504] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa82b1e5f6f4f54b06b6d1b91c553ab" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37177 } }
I20260925 10:59:18.305229 18396 leader_election.cc:302] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: faa82b1e5f6f4f54b06b6d1b91c553ab; no voters:
I20260925 10:59:18.305893 18396 leader_election.cc:288] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:18.306080 18401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:18.306845 18401 raft_consensus.cc:686] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [term 1 LEADER]: Becoming Leader. State: Replica: faa82b1e5f6f4f54b06b6d1b91c553ab, State: Running, Role: LEADER
I20260925 10:59:18.307364 18401 consensus_queue.cc:237] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa82b1e5f6f4f54b06b6d1b91c553ab" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37177 } }
I20260925 10:59:18.308189 18396 sys_catalog.cc:564] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:18.310211 18403 sys_catalog.cc:455] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader faa82b1e5f6f4f54b06b6d1b91c553ab. Latest consensus state: current_term: 1 leader_uuid: "faa82b1e5f6f4f54b06b6d1b91c553ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa82b1e5f6f4f54b06b6d1b91c553ab" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37177 } } }
I20260925 10:59:18.310634 18403 sys_catalog.cc:458] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:18.310912 18402 sys_catalog.cc:455] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "faa82b1e5f6f4f54b06b6d1b91c553ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa82b1e5f6f4f54b06b6d1b91c553ab" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 37177 } } }
I20260925 10:59:18.311307 18402 sys_catalog.cc:458] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:18.313201 18408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:18.318677 18408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:18.328912 18408 catalog_manager.cc:1347] Generated new cluster ID: daf6c24c8aeb4ffdb14cc6b6198d99bf
I20260925 10:59:18.329057 18408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:18.371899 18408 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:18.373339 18408 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:18.381817 18408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab: Generated new TSK 0
I20260925 10:59:18.382699 18408 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:18.396188 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:37177
--builtin_ntp_servers=127.31.62.148:44859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 10:59:18.730890 18420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:18.731199 18420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:18.731321 18420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:18.738685 18420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:18.738816 18420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:18.738961 18420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:59:18.748288 18420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:37177
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:18.749661 18420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:18.751448 18420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:18.764528 18425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:18.764528 18426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:18.764535 18428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:18.765414 18420 server_base.cc:1034] running on GCE node
I20260925 10:59:18.765988 18420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:18.766906 18420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:18.768119 18420 hybrid_clock.cc:648] HybridClock initialized: now 1790333958768056 us; error 51 us; skew 500 ppm
I20260925 10:59:18.768453 18420 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:59:18.770725 18420 webserver.cc:460] Webserver started at http://127.31.62.129:42303/ using document root <none> and password file <none>
I20260925 10:59:18.771472 18420 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:18.771566 18420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:18.771885 18420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:18.774838 18420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "86bf12a51078469eb2bc51b301aba6f2"
format_stamp: "Formatted at 2026-09-25 10:59:18 on dist-test-slave-nx7s"
server_key: "c9930b4ffc8502a7a2ff37846b887bdd"
server_key_iv: "0182e06bc3e155d0fefb62612a6f1df8"
server_key_version: "encryptionkey@0"
I20260925 10:59:18.775655 18420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "86bf12a51078469eb2bc51b301aba6f2"
format_stamp: "Formatted at 2026-09-25 10:59:18 on dist-test-slave-nx7s"
server_key: "c9930b4ffc8502a7a2ff37846b887bdd"
server_key_iv: "0182e06bc3e155d0fefb62612a6f1df8"
server_key_version: "encryptionkey@0"
I20260925 10:59:18.781069 18420 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260925 10:59:18.785012 18434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:18.786667 18420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:59:18.786891 18420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "86bf12a51078469eb2bc51b301aba6f2"
format_stamp: "Formatted at 2026-09-25 10:59:18 on dist-test-slave-nx7s"
server_key: "c9930b4ffc8502a7a2ff37846b887bdd"
server_key_iv: "0182e06bc3e155d0fefb62612a6f1df8"
server_key_version: "encryptionkey@0"
I20260925 10:59:18.787139 18420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:18.819046 18420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:18.820025 18420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:18.820331 18420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:18.821383 18420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:18.822945 18420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:18.823046 18420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:18.823153 18420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:18.823218 18420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:18.852365 18420 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:46473
I20260925 10:59:18.852463 18546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:46473 every 8 connection(s)
I20260925 10:59:18.853794 18420 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:59:18.863772 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 18420
I20260925 10:59:18.863943 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:59:18.864388 31994 external_mini_cluster.cc:1428] Setting key e3b92165d6af288d88d51dae41a251f7
I20260925 10:59:18.866714 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:18.856491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:39925 (local address 127.31.62.190:37177)
0925 10:59:18.856983 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:18.856988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:18.858263 (+ 1275us) server_negotiation.cc:407] Connection header received
0925 10:59:18.860135 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:18.860140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:18.860223 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:18.860320 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:18.861495 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:18.862239 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:18.863784 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:18.864237 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:18.864350 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:18.865871 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:18.865875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:18.865879 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:18.865897 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:18.865940 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:18.865952 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:18.866033 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:18.866320 (+ 287us) server_negotiation.cc:299] Negotiation successful
0925 10:59:18.866417 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":229,"threads_started":1}
I20260925 10:59:18.867509 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:37177
--builtin_ntp_servers=127.31.62.148:44859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 10:59:18.867954 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:18.857167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:37177 (local address 127.31.62.129:39925)
0925 10:59:18.858093 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:18.858143 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:18.859847 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:18.860598 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:18.860607 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:18.860703 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:18.861323 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:18.861340 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:18.862390 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:18.862394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:18.863646 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:18.863658 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:18.864466 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:18.864473 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:18.864717 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:18.865513 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:59:18.865517 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:18.865573 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:59:18.865677 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:18.866048 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:18.866060 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:18.866068 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 10:59:18.866254 (+ 186us) client_negotiation.cc:240] Negotiation successful
0925 10:59:18.866671 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":279,"threads_started":1}
I20260925 10:59:18.869565 18547 heartbeater.cc:347] Connected to a master server at 127.31.62.190:37177
I20260925 10:59:18.869895 18547 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:18.870770 18547 heartbeater.cc:511] Master 127.31.62.190:37177 requested a full tablet report, sending...
I20260925 10:59:18.873004 18361 ts_manager.cc:194] Registered new tserver with Master: 86bf12a51078469eb2bc51b301aba6f2 (127.31.62.129:46473)
I20260925 10:59:18.874532 18361 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.62.129:39925
W20260925 10:59:19.192008 18551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:19.192291 18551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:19.192386 18551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:19.199725 18551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:19.199848 18551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:19.199991 18551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:59:19.209506 18551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:37177
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:19.210862 18551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:19.212589 18551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:19.226133 18556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:19.226097 18557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:19.226066 18559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:19.227078 18551 server_base.cc:1034] running on GCE node
I20260925 10:59:19.227798 18551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:19.228976 18551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:19.230233 18551 hybrid_clock.cc:648] HybridClock initialized: now 1790333959230174 us; error 53 us; skew 500 ppm
I20260925 10:59:19.230620 18551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:59:19.233124 18551 webserver.cc:460] Webserver started at http://127.31.62.130:35861/ using document root <none> and password file <none>
I20260925 10:59:19.233882 18551 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:19.233997 18551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:19.234308 18551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:19.237401 18551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "d721453f9c4a48568345c72e1a57e14c"
format_stamp: "Formatted at 2026-09-25 10:59:19 on dist-test-slave-nx7s"
server_key: "d0fe87cde39f6a787ea58f73e9e410d3"
server_key_iv: "320a888a220416e8e64fb7e604aa4f85"
server_key_version: "encryptionkey@0"
I20260925 10:59:19.238224 18551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "d721453f9c4a48568345c72e1a57e14c"
format_stamp: "Formatted at 2026-09-25 10:59:19 on dist-test-slave-nx7s"
server_key: "d0fe87cde39f6a787ea58f73e9e410d3"
server_key_iv: "320a888a220416e8e64fb7e604aa4f85"
server_key_version: "encryptionkey@0"
I20260925 10:59:19.243687 18551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260925 10:59:19.247838 18565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:19.249435 18551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:59:19.249615 18551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "d721453f9c4a48568345c72e1a57e14c"
format_stamp: "Formatted at 2026-09-25 10:59:19 on dist-test-slave-nx7s"
server_key: "d0fe87cde39f6a787ea58f73e9e410d3"
server_key_iv: "320a888a220416e8e64fb7e604aa4f85"
server_key_version: "encryptionkey@0"
I20260925 10:59:19.249830 18551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:19.269718 18551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:19.270553 18551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:19.270839 18551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:19.271832 18551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:19.273377 18551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:19.273470 18551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:19.273574 18551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:19.273640 18551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:19.306398 18551 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:39119
I20260925 10:59:19.306495 18677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:39119 every 8 connection(s)
I20260925 10:59:19.307693 18551 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:59:19.312258 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 18551
I20260925 10:59:19.312458 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:59:19.313046 31994 external_mini_cluster.cc:1428] Setting key fad4ade7c9b54052548fa559c3ce3af9
I20260925 10:59:19.316169 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:37177
--builtin_ntp_servers=127.31.62.148:44859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 10:59:19.321686 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:19.310349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:51009 (local address 127.31.62.190:37177)
0925 10:59:19.310522 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:19.310526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:19.311971 (+ 1445us) server_negotiation.cc:407] Connection header received
0925 10:59:19.314349 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:19.314355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:19.314448 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:19.314531 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:19.315894 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.316689 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.318884 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.319281 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.319397 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:19.320994 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:19.320999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:19.321002 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:19.321023 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:19.321056 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:19.321069 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:19.321144 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:19.321367 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 10:59:19.321452 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 10:59:19.323097 18680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:19.310917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:37177 (local address 127.31.62.130:51009)
0925 10:59:19.311792 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:19.311836 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:19.314059 (+ 2223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:19.314787 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:19.314797 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:19.314903 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:19.315704 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:19.315729 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.316841 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.316873 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:19.318729 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:19.318744 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.319497 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.319504 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:19.319781 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:19.320555 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:59:19.320559 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:19.320614 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:59:19.320721 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:19.321155 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:19.321165 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:19.321173 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 10:59:19.321360 (+ 187us) client_negotiation.cc:240] Negotiation successful
0925 10:59:19.321789 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":250,"threads_started":1}
I20260925 10:59:19.324635 18678 heartbeater.cc:347] Connected to a master server at 127.31.62.190:37177
I20260925 10:59:19.325018 18678 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:19.325973 18678 heartbeater.cc:511] Master 127.31.62.190:37177 requested a full tablet report, sending...
I20260925 10:59:19.327526 18361 ts_manager.cc:194] Registered new tserver with Master: d721453f9c4a48568345c72e1a57e14c (127.31.62.130:39119)
I20260925 10:59:19.328454 18361 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.62.130:51009
W20260925 10:59:19.655525 18681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:19.655861 18681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:19.655992 18681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:19.663769 18681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:19.663902 18681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:19.664064 18681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:59:19.673794 18681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:37177
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:19.675223 18681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:19.677187 18681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:19.691726 18686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:19.691622 18687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:19.691649 18689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:19.692351 18681 server_base.cc:1034] running on GCE node
I20260925 10:59:19.693192 18681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:19.694250 18681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:19.695477 18681 hybrid_clock.cc:648] HybridClock initialized: now 1790333959695416 us; error 53 us; skew 500 ppm
I20260925 10:59:19.695816 18681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:59:19.698359 18681 webserver.cc:460] Webserver started at http://127.31.62.131:45497/ using document root <none> and password file <none>
I20260925 10:59:19.699129 18681 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:19.699219 18681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:19.699497 18681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:19.702487 18681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "d85dd375ed884f8e8c28d5af00c5ad86"
format_stamp: "Formatted at 2026-09-25 10:59:19 on dist-test-slave-nx7s"
server_key: "9d24345bda88dee19747c1bb9a11138c"
server_key_iv: "01102b8783f1a1d83ad89a2a605b072a"
server_key_version: "encryptionkey@0"
I20260925 10:59:19.703279 18681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "d85dd375ed884f8e8c28d5af00c5ad86"
format_stamp: "Formatted at 2026-09-25 10:59:19 on dist-test-slave-nx7s"
server_key: "9d24345bda88dee19747c1bb9a11138c"
server_key_iv: "01102b8783f1a1d83ad89a2a605b072a"
server_key_version: "encryptionkey@0"
I20260925 10:59:19.708619 18681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 10:59:19.712504 18695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:19.714322 18681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:59:19.714509 18681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "d85dd375ed884f8e8c28d5af00c5ad86"
format_stamp: "Formatted at 2026-09-25 10:59:19 on dist-test-slave-nx7s"
server_key: "9d24345bda88dee19747c1bb9a11138c"
server_key_iv: "01102b8783f1a1d83ad89a2a605b072a"
server_key_version: "encryptionkey@0"
I20260925 10:59:19.714725 18681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:19.729959 18681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:19.730794 18681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:19.731079 18681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:19.732059 18681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:19.733673 18681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:19.733770 18681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:19.733896 18681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:19.733963 18681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:19.764035 18681 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:36213
I20260925 10:59:19.764062 18807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:36213 every 8 connection(s)
I20260925 10:59:19.765404 18681 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:59:19.774381 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 18681
I20260925 10:59:19.774546 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:59:19.774992 31994 external_mini_cluster.cc:1428] Setting key b70e1e71f0a2f4cbbd6deb91b03b39a6
I20260925 10:59:19.778945 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:19.768144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:52441 (local address 127.31.62.190:37177)
0925 10:59:19.768327 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:19.768331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:19.769912 (+ 1581us) server_negotiation.cc:407] Connection header received
0925 10:59:19.771684 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:19.771690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:19.771777 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:19.771868 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:19.773164 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.773995 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.775839 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.776435 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.776548 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:19.778146 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:19.778152 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:19.778156 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:19.778176 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:19.778209 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:19.778229 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:19.778310 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:19.778530 (+ 220us) server_negotiation.cc:299] Negotiation successful
0925 10:59:19.778631 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 10:59:19.780025 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:19.768765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:37177 (local address 127.31.62.131:52441)
0925 10:59:19.769749 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:19.769786 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:19.771388 (+ 1602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:19.772138 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:19.772147 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:19.772241 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:19.772835 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:19.772889 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.774168 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.774173 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:19.775695 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:19.775710 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.776666 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.776673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:19.776961 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:19.777820 (+ 859us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:59:19.777822 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:19.777872 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:59:19.777960 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:19.778316 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:19.778328 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:19.778336 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 10:59:19.778496 (+ 160us) client_negotiation.cc:240] Negotiation successful
0925 10:59:19.778937 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":274,"threads_started":1}
I20260925 10:59:19.781543 18808 heartbeater.cc:347] Connected to a master server at 127.31.62.190:37177
I20260925 10:59:19.781924 18808 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:19.782719 18808 heartbeater.cc:511] Master 127.31.62.190:37177 requested a full tablet report, sending...
I20260925 10:59:19.784274 18361 ts_manager.cc:194] Registered new tserver with Master: d85dd375ed884f8e8c28d5af00c5ad86 (127.31.62.131:36213)
I20260925 10:59:19.785178 18361 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.62.131:52441
I20260925 10:59:19.790707 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:59:19.800966 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:19.794991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50152 (local address 127.31.62.190:37177)
0925 10:59:19.795187 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:19.795194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:19.795653 (+ 459us) server_negotiation.cc:407] Connection header received
0925 10:59:19.795844 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:19.795849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:19.795914 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:19.796005 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:19.796567 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.797359 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.798773 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.799185 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.799278 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:19.800270 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:19.800275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:19.800278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:19.800297 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:19.800328 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:19.800340 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:19.800473 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:19.800598 (+ 125us) server_negotiation.cc:299] Negotiation successful
0925 10:59:19.800647 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 10:59:19.804874 18361 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:59:19.811851 18361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:59:19.836091 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:19.824377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:36213 (local address 127.0.0.1:50312)
0925 10:59:19.827212 (+ 2835us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:19.827230 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:19.827350 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:19.828275 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:19.828279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:19.828304 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:19.828646 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:19.828658 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.830460 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.830465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:19.832833 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:19.832868 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.835097 (+ 2229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.835103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:19.835319 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:19.835371 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 10:59:19.835516 (+ 145us) client_negotiation.cc:240] Negotiation successful
0925 10:59:19.835760 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2691,"thread_start_us":915,"threads_started":1}
I20260925 10:59:19.837064 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:19.824980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50312 (local address 127.31.62.131:36213)
0925 10:59:19.826384 (+ 1404us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:19.826391 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:19.827492 (+ 1101us) server_negotiation.cc:407] Connection header received
0925 10:59:19.827592 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:19.827600 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:19.827861 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:19.828025 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:19.828832 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.830295 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.833167 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.834578 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.834691 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:19.834969 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:19.836506 (+ 1537us) server_negotiation.cc:299] Negotiation successful
0925 10:59:19.836692 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1257,"thread_start_us":1152,"threads_started":1}
I20260925 10:59:19.838562 18821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:19.824272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:39119 (local address 127.0.0.1:57616)
0925 10:59:19.825199 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:19.825252 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:19.825472 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:19.827194 (+ 1722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:19.827201 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:19.827235 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:19.827737 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:19.827750 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.830936 (+ 3186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.830941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:19.834297 (+ 3356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:19.834311 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.836390 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.836396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:19.836570 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:19.836602 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 10:59:19.836680 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 10:59:19.836807 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":2681,"threads_started":1}
I20260925 10:59:19.839287 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:19.824396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57616 (local address 127.31.62.130:39119)
0925 10:59:19.826091 (+ 1695us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:19.826101 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:19.826536 (+ 435us) server_negotiation.cc:407] Connection header received
0925 10:59:19.826657 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:19.826666 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:19.826962 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:19.827164 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:19.827965 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.830779 (+ 2814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.834518 (+ 3739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.835896 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.836027 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:19.836273 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:19.838779 (+ 2506us) server_negotiation.cc:299] Negotiation successful
0925 10:59:19.838975 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1508,"thread_start_us":1393,"threads_started":1}
I20260925 10:59:19.841584 18743 tablet_service.cc:1467] Processing CreateTablet for tablet 53b21042d0ba462188d586f0d5f8528a (DEFAULT_TABLE table=test-table [id=8c9a40bfad13434b9e206843c984147c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:19.843317 18743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53b21042d0ba462188d586f0d5f8528a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:19.845088 18613 tablet_service.cc:1467] Processing CreateTablet for tablet 53b21042d0ba462188d586f0d5f8528a (DEFAULT_TABLE table=test-table [id=8c9a40bfad13434b9e206843c984147c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:19.845943 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:19.827576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:46473 (local address 127.0.0.1:42026)
0925 10:59:19.837511 (+ 9935us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:19.837528 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:19.837624 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:19.839391 (+ 1767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:19.839397 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:19.839427 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:19.839609 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:19.839619 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.841775 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.841780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:19.843371 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:19.843387 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.845166 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.845172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:19.845363 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:19.845398 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 10:59:19.845500 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 10:59:19.845649 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9786,"thread_start_us":13469,"threads_started":1}
I20260925 10:59:19.846082 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:19.827610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42026 (local address 127.31.62.129:46473)
0925 10:59:19.838450 (+ 10840us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:19.838460 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:19.838495 (+ 35us) server_negotiation.cc:407] Connection header received
0925 10:59:19.838610 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:19.838618 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:19.838914 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:19.839799 (+ 885us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:19.840027 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.841612 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.843560 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:19.844998 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:19.845143 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:19.845380 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:19.845555 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 10:59:19.845734 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10628,"thread_start_us":11484,"threads_started":1}
I20260925 10:59:19.846899 18613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53b21042d0ba462188d586f0d5f8528a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:19.850255 18482 tablet_service.cc:1467] Processing CreateTablet for tablet 53b21042d0ba462188d586f0d5f8528a (DEFAULT_TABLE table=test-table [id=8c9a40bfad13434b9e206843c984147c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:19.851689 18482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53b21042d0ba462188d586f0d5f8528a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:19.863296 18826 tablet_bootstrap.cc:492] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86: Bootstrap starting.
I20260925 10:59:19.864485 18827 tablet_bootstrap.cc:492] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c: Bootstrap starting.
I20260925 10:59:19.868664 18826 tablet_bootstrap.cc:654] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:19.868677 18828 tablet_bootstrap.cc:492] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2: Bootstrap starting.
I20260925 10:59:19.869071 18827 tablet_bootstrap.cc:654] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:19.869953 18826 log.cc:824] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:19.870371 18827 log.cc:824] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:19.872602 18828 tablet_bootstrap.cc:654] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:19.873006 18827 tablet_bootstrap.cc:492] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c: No bootstrap required, opened a new log
I20260925 10:59:19.873147 18826 tablet_bootstrap.cc:492] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86: No bootstrap required, opened a new log
I20260925 10:59:19.873267 18827 ts_tablet_manager.cc:1397] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260925 10:59:19.873425 18826 ts_tablet_manager.cc:1397] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260925 10:59:19.873874 18828 log.cc:824] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:19.876938 18828 tablet_bootstrap.cc:492] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2: No bootstrap required, opened a new log
I20260925 10:59:19.877259 18828 ts_tablet_manager.cc:1397] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260925 10:59:19.877744 18547 heartbeater.cc:503] Master 127.31.62.190:37177 was elected leader, sending a full tablet report...
I20260925 10:59:19.883328 18827 raft_consensus.cc:348] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d721453f9c4a48568345c72e1a57e14c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39119 } } peers { permanent_uuid: "d85dd375ed884f8e8c28d5af00c5ad86" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36213 } } peers { permanent_uuid: "86bf12a51078469eb2bc51b301aba6f2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 46473 } }
I20260925 10:59:19.883348 18826 raft_consensus.cc:348] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d721453f9c4a48568345c72e1a57e14c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39119 } } peers { permanent_uuid: "d85dd375ed884f8e8c28d5af00c5ad86" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36213 } } peers { permanent_uuid: "86bf12a51078469eb2bc51b301aba6f2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 46473 } }
I20260925 10:59:19.883715 18827 raft_consensus.cc:374] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:19.883759 18826 raft_consensus.cc:374] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:19.883819 18827 raft_consensus.cc:729] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d721453f9c4a48568345c72e1a57e14c, State: Initialized, Role: FOLLOWER
I20260925 10:59:19.883898 18826 raft_consensus.cc:729] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d85dd375ed884f8e8c28d5af00c5ad86, State: Initialized, Role: FOLLOWER
I20260925 10:59:19.884424 18827 consensus_queue.cc:260] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d721453f9c4a48568345c72e1a57e14c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39119 } } peers { permanent_uuid: "d85dd375ed884f8e8c28d5af00c5ad86" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36213 } } peers { permanent_uuid: "86bf12a51078469eb2bc51b301aba6f2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 46473 } }
I20260925 10:59:19.884593 18826 consensus_queue.cc:260] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d721453f9c4a48568345c72e1a57e14c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39119 } } peers { permanent_uuid: "d85dd375ed884f8e8c28d5af00c5ad86" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36213 } } peers { permanent_uuid: "86bf12a51078469eb2bc51b301aba6f2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 46473 } }
I20260925 10:59:19.886670 18808 heartbeater.cc:503] Master 127.31.62.190:37177 was elected leader, sending a full tablet report...
I20260925 10:59:19.887373 18826 ts_tablet_manager.cc:1428] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260925 10:59:19.887647 18828 raft_consensus.cc:348] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d721453f9c4a48568345c72e1a57e14c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39119 } } peers { permanent_uuid: "d85dd375ed884f8e8c28d5af00c5ad86" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36213 } } peers { permanent_uuid: "86bf12a51078469eb2bc51b301aba6f2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 46473 } }
I20260925 10:59:19.887980 18828 raft_consensus.cc:374] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:19.888101 18828 raft_consensus.cc:729] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86bf12a51078469eb2bc51b301aba6f2, State: Initialized, Role: FOLLOWER
I20260925 10:59:19.888679 18828 consensus_queue.cc:260] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d721453f9c4a48568345c72e1a57e14c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39119 } } peers { permanent_uuid: "d85dd375ed884f8e8c28d5af00c5ad86" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36213 } } peers { permanent_uuid: "86bf12a51078469eb2bc51b301aba6f2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 46473 } }
I20260925 10:59:19.890430 18828 ts_tablet_manager.cc:1428] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2: Time spent starting tablet: real 0.013s user 0.005s sys 0.007s
I20260925 10:59:19.891371 18678 heartbeater.cc:503] Master 127.31.62.190:37177 was elected leader, sending a full tablet report...
I20260925 10:59:19.892369 18827 ts_tablet_manager.cc:1428] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c: Time spent starting tablet: real 0.019s user 0.013s sys 0.004s
W20260925 10:59:20.017269 18809 tablet.cc:2369] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:59:20.059716 18679 tablet.cc:2369] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:59:20.106290 18548 tablet.cc:2369] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:59:20.167765 18834 raft_consensus.cc:482] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:59:20.168133 18834 raft_consensus.cc:504] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d721453f9c4a48568345c72e1a57e14c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39119 } } peers { permanent_uuid: "d85dd375ed884f8e8c28d5af00c5ad86" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36213 } } peers { permanent_uuid: "86bf12a51078469eb2bc51b301aba6f2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 46473 } }
I20260925 10:59:20.169898 18834 leader_election.cc:288] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d721453f9c4a48568345c72e1a57e14c (127.31.62.130:39119), d85dd375ed884f8e8c28d5af00c5ad86 (127.31.62.131:36213)
I20260925 10:59:20.176810 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:20.170357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:54227 (local address 127.31.62.130:39119)
0925 10:59:20.170552 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:20.170559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:20.171217 (+ 658us) server_negotiation.cc:407] Connection header received
0925 10:59:20.171393 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:20.171397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:20.171489 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:20.171561 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:20.172163 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.172830 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.174896 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.175711 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.175860 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:20.175929 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:20.176408 (+ 479us) server_negotiation.cc:299] Negotiation successful
0925 10:59:20.176550 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 10:59:20.176832 18835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:20.170162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:39119 (local address 127.31.62.129:54227)
0925 10:59:20.171100 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:20.171119 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:20.171266 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:20.171596 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:20.171601 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:20.171622 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:20.172031 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:20.172040 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.172983 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.172988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:20.174693 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:20.174704 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.176045 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.176052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:20.176219 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:20.176247 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 10:59:20.176334 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 10:59:20.176473 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"mutex_wait_us":106,"thread_start_us":566,"threads_started":1}
I20260925 10:59:20.177883 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:20.170503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:49331 (local address 127.31.62.131:36213)
0925 10:59:20.170676 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:20.170682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:20.171255 (+ 573us) server_negotiation.cc:407] Connection header received
0925 10:59:20.171418 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:20.171421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:20.171479 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:20.171549 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:20.172145 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.172937 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.174870 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.175694 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.175859 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:20.175928 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:20.177533 (+ 1605us) server_negotiation.cc:299] Negotiation successful
0925 10:59:20.177671 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 10:59:20.178182 18836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:20.170313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:36213 (local address 127.31.62.129:49331)
0925 10:59:20.171153 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:20.171167 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:20.171286 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:20.171574 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:20.171578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:20.171607 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:20.171975 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:20.171987 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.173061 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.173067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:20.174686 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:20.174698 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.175900 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.175908 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:20.176160 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:20.177009 (+ 849us) client_negotiation.cc:769] Sending connection context
0925 10:59:20.177143 (+ 134us) client_negotiation.cc:240] Negotiation successful
0925 10:59:20.177271 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":843,"thread_start_us":450,"threads_started":1}
I20260925 10:59:20.177976 18633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53b21042d0ba462188d586f0d5f8528a" candidate_uuid: "86bf12a51078469eb2bc51b301aba6f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d721453f9c4a48568345c72e1a57e14c" is_pre_election: true
I20260925 10:59:20.178597 18633 raft_consensus.cc:2454] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86bf12a51078469eb2bc51b301aba6f2 in term 0.
I20260925 10:59:20.179221 18763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53b21042d0ba462188d586f0d5f8528a" candidate_uuid: "86bf12a51078469eb2bc51b301aba6f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d85dd375ed884f8e8c28d5af00c5ad86" is_pre_election: true
I20260925 10:59:20.179430 18436 leader_election.cc:302] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86bf12a51078469eb2bc51b301aba6f2, d721453f9c4a48568345c72e1a57e14c; no voters:
I20260925 10:59:20.179858 18834 raft_consensus.cc:2781] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:59:20.179826 18763 raft_consensus.cc:2454] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86bf12a51078469eb2bc51b301aba6f2 in term 0.
I20260925 10:59:20.179992 18834 raft_consensus.cc:482] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:59:20.180090 18834 raft_consensus.cc:3037] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:20.182256 18834 raft_consensus.cc:504] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d721453f9c4a48568345c72e1a57e14c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39119 } } peers { permanent_uuid: "d85dd375ed884f8e8c28d5af00c5ad86" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36213 } } peers { permanent_uuid: "86bf12a51078469eb2bc51b301aba6f2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 46473 } }
I20260925 10:59:20.182837 18834 leader_election.cc:288] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [CANDIDATE]: Term 1 election: Requested vote from peers d721453f9c4a48568345c72e1a57e14c (127.31.62.130:39119), d85dd375ed884f8e8c28d5af00c5ad86 (127.31.62.131:36213)
I20260925 10:59:20.183317 18763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53b21042d0ba462188d586f0d5f8528a" candidate_uuid: "86bf12a51078469eb2bc51b301aba6f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d85dd375ed884f8e8c28d5af00c5ad86"
I20260925 10:59:20.183511 18763 raft_consensus.cc:3037] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:20.185693 18763 raft_consensus.cc:2454] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86bf12a51078469eb2bc51b301aba6f2 in term 1.
I20260925 10:59:20.186058 18633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53b21042d0ba462188d586f0d5f8528a" candidate_uuid: "86bf12a51078469eb2bc51b301aba6f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d721453f9c4a48568345c72e1a57e14c"
I20260925 10:59:20.186106 18435 leader_election.cc:302] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86bf12a51078469eb2bc51b301aba6f2, d85dd375ed884f8e8c28d5af00c5ad86; no voters:
I20260925 10:59:20.186265 18633 raft_consensus.cc:3037] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:20.186379 18834 raft_consensus.cc:2781] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:20.186863 18834 raft_consensus.cc:686] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [term 1 LEADER]: Becoming Leader. State: Replica: 86bf12a51078469eb2bc51b301aba6f2, State: Running, Role: LEADER
I20260925 10:59:20.187394 18834 consensus_queue.cc:237] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d721453f9c4a48568345c72e1a57e14c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39119 } } peers { permanent_uuid: "d85dd375ed884f8e8c28d5af00c5ad86" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36213 } } peers { permanent_uuid: "86bf12a51078469eb2bc51b301aba6f2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 46473 } }
I20260925 10:59:20.188768 18633 raft_consensus.cc:2454] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86bf12a51078469eb2bc51b301aba6f2 in term 1.
I20260925 10:59:20.191866 18361 catalog_manager.cc:5495] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86bf12a51078469eb2bc51b301aba6f2 (127.31.62.129). New cstate: current_term: 1 leader_uuid: "86bf12a51078469eb2bc51b301aba6f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d721453f9c4a48568345c72e1a57e14c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d85dd375ed884f8e8c28d5af00c5ad86" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86bf12a51078469eb2bc51b301aba6f2" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 46473 } health_report { overall_health: HEALTHY } } }
I20260925 10:59:20.295562 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:20.289581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.31.62.129:46473)
0925 10:59:20.289808 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:20.289812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:20.289831 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:59:20.290029 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:20.290033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:20.290124 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:20.290252 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:59:20.290694 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.291486 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.293078 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.293510 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.293615 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:20.294240 (+ 625us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:59:20.294772 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:59:20.294925 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:20.295152 (+ 227us) server_negotiation.cc:299] Negotiation successful
0925 10:59:20.295204 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 10:59:20.303905 18763 raft_consensus.cc:1261] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86 [term 1 FOLLOWER]: Refusing update from remote peer 86bf12a51078469eb2bc51b301aba6f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:59:20.303947 18633 raft_consensus.cc:1261] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c [term 1 FOLLOWER]: Refusing update from remote peer 86bf12a51078469eb2bc51b301aba6f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:59:20.304888 18837 consensus_queue.cc:1035] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d85dd375ed884f8e8c28d5af00c5ad86" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:59:20.305248 18834 consensus_queue.cc:1035] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d721453f9c4a48568345c72e1a57e14c" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 39119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:59:20.322193 18840 mvcc.cc:205] Tried to move back new op lower bound from 7333207901395771392 to 7333207900931268608. Current Snapshot: MvccSnapshot[applied={T|T < 7333207901395771392}]
I20260925 10:59:20.323534 18841 mvcc.cc:205] Tried to move back new op lower bound from 7333207901395771392 to 7333207900931268608. Current Snapshot: MvccSnapshot[applied={T|T < 7333207901395771392}]
I20260925 10:59:20.336809 18361 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:59:20.337138 18361 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:59:20.340394 18361 catalog_manager.cc:5782] T 00000000000000000000000000000000 P faa82b1e5f6f4f54b06b6d1b91c553ab: Sending DeleteTablet for 3 replicas of tablet 53b21042d0ba462188d586f0d5f8528a
I20260925 10:59:20.341699 18613 tablet_service.cc:1514] Processing DeleteTablet for tablet 53b21042d0ba462188d586f0d5f8528a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:20 UTC) from {username='slave'} at 127.0.0.1:57616
I20260925 10:59:20.342027 18482 tablet_service.cc:1514] Processing DeleteTablet for tablet 53b21042d0ba462188d586f0d5f8528a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:20 UTC) from {username='slave'} at 127.0.0.1:42026
I20260925 10:59:20.342474 18743 tablet_service.cc:1514] Processing DeleteTablet for tablet 53b21042d0ba462188d586f0d5f8528a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:20 UTC) from {username='slave'} at 127.0.0.1:50312
I20260925 10:59:20.342692 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 18420
I20260925 10:59:20.354954 18851 tablet_replica.cc:330] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86: stopping tablet replica
I20260925 10:59:20.355504 18851 raft_consensus.cc:2229] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:59:20.356068 18851 raft_consensus.cc:2258] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:20.356914 18849 tablet_replica.cc:330] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c: stopping tablet replica
I20260925 10:59:20.357326 18849 raft_consensus.cc:2229] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:59:20.357868 18849 raft_consensus.cc:2258] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:20.358299 18850 tablet_replica.cc:330] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2: stopping tablet replica
I20260925 10:59:20.358685 18850 raft_consensus.cc:2229] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:59:20.359332 18850 raft_consensus.cc:2258] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:20.360451 18851 ts_tablet_manager.cc:1905] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:20.361685 18849 ts_tablet_manager.cc:1905] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:20.361997 18850 ts_tablet_manager.cc:1905] T 53b21042d0ba462188d586f0d5f8528a P 86bf12a51078469eb2bc51b301aba6f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:20.367326 18851 ts_tablet_manager.cc:1918] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:20.367509 18851 log.cc:1196] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/53b21042d0ba462188d586f0d5f8528a
I20260925 10:59:20.367553 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:20.344045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42050 (local address 127.31.62.129:46473)
0925 10:59:20.344646 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:20.344653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:20.354025 (+ 9372us) server_negotiation.cc:407] Connection header received
0925 10:59:20.354266 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:20.354272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:20.354399 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:20.354504 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:20.355606 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.356863 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.359410 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.360314 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.360440 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:20.365676 (+ 5236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:20.365690 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:20.365713 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:20.366564 (+ 851us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:20.366629 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:20.366656 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:20.366737 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:20.367008 (+ 271us) server_negotiation.cc:299] Negotiation successful
0925 10:59:20.367122 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481}
I20260925 10:59:20.367954 18851 ts_tablet_manager.cc:1939] T 53b21042d0ba462188d586f0d5f8528a P d85dd375ed884f8e8c28d5af00c5ad86: Deleting consensus metadata
I20260925 10:59:20.368228 18849 ts_tablet_manager.cc:1918] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:20.368381 18849 log.cc:1196] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/53b21042d0ba462188d586f0d5f8528a
I20260925 10:59:20.368897 18849 ts_tablet_manager.cc:1939] T 53b21042d0ba462188d586f0d5f8528a P d721453f9c4a48568345c72e1a57e14c: Deleting consensus metadata
I20260925 10:59:20.369118 18542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:20.369896 18345 catalog_manager.cc:4841] TS d85dd375ed884f8e8c28d5af00c5ad86 (127.31.62.131:36213): tablet 53b21042d0ba462188d586f0d5f8528a (table test-table [id=8c9a40bfad13434b9e206843c984147c]) successfully deleted
I20260925 10:59:20.370594 18346 catalog_manager.cc:4841] TS d721453f9c4a48568345c72e1a57e14c (127.31.62.130:39119): tablet 53b21042d0ba462188d586f0d5f8528a (table test-table [id=8c9a40bfad13434b9e206843c984147c]) successfully deleted
I20260925 10:59:20.467809 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 18420
W20260925 10:59:20.483026 18346 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.62.129:46473 (error 108)
I20260925 10:59:20.483225 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 18551
I20260925 10:59:20.485143 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:20.484347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:46473 (local address 127.0.0.1:42052)
0925 10:59:20.484798 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:20.484961 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:46473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":215,"threads_started":1}
W20260925 10:59:20.485370 18346 catalog_manager.cc:4584] TS 86bf12a51078469eb2bc51b301aba6f2 (127.31.62.129:46473): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 53b21042d0ba462188d586f0d5f8528a: Network error: Client connection negotiation failed: client connection to 127.31.62.129:46473: connect: Connection refused (error 111)
I20260925 10:59:20.490085 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:20.484058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57632 (local address 127.31.62.130:39119)
0925 10:59:20.484223 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:20.484227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:20.484245 (+ 18us) server_negotiation.cc:407] Connection header received
0925 10:59:20.484442 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:20.484446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:20.484546 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:20.484632 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:20.485094 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.485878 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.487125 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.487516 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.487619 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:20.488603 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:20.488612 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:20.488628 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:20.489359 (+ 731us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:20.489407 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:20.489434 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:20.489530 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:20.489673 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 10:59:20.489774 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 10:59:20.490738 18673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:20.539065 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:20.538621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:46473 (local address 127.0.0.1:42068)
0925 10:59:20.538818 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:20.538908 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:46473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260925 10:59:20.540059 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:20.539683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:46473 (local address 127.0.0.1:42070)
0925 10:59:20.539827 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:20.539910 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:46473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 10:59:20.579303 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 18551
I20260925 10:59:20.593453 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 18681
I20260925 10:59:20.600208 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:20.594183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50316 (local address 127.31.62.131:36213)
0925 10:59:20.594341 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:20.594345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:20.594366 (+ 21us) server_negotiation.cc:407] Connection header received
0925 10:59:20.594522 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:20.594526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:20.594613 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:20.594708 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:20.595071 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.595799 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.597102 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:20.597526 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:20.597632 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:20.598648 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:20.598661 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:20.598683 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:20.599431 (+ 748us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:20.599484 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:20.599504 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:20.599607 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:20.599737 (+ 130us) server_negotiation.cc:299] Negotiation successful
0925 10:59:20.599885 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 10:59:20.600442 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:20.600010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:46473 (local address 127.0.0.1:42084)
0925 10:59:20.600194 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:20.600270 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:46473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 10:59:20.600826 18803 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:20.601643 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:20.601187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:46473 (local address 127.0.0.1:42086)
0925 10:59:20.601375 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:20.601482 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:46473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260925 10:59:20.682349 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:20.681742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:46473 (local address 127.0.0.1:42094)
0925 10:59:20.682055 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:20.682170 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:46473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260925 10:59:20.683547 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:20.683056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:46473 (local address 127.0.0.1:42098)
0925 10:59:20.683298 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:20.683382 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.129:46473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260925 10:59:20.689563 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 18681
I20260925 10:59:20.703437 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 18330
I20260925 10:59:20.704559 18391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:20.811146 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 18330
2026-09-25T10:59:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3048 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:20 dist-test-slave-nx7s krb5kdc[18861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:20 dist-test-slave-nx7s krb5kdc[18861](info): set up 2 sockets
Sep 25 10:59:20 dist-test-slave-nx7s krb5kdc[18861](info): commencing operation
krb5kdc: starting...
W20260925 10:59:21.833279 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:21 dist-test-slave-nx7s krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:59:22.076552 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:38791
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:38791
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:22.407054 18874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:22.407367 18874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:22.407459 18874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:22.414913 18874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:22.415045 18874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:22.415093 18874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:22.415153 18874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:22.415196 18874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:22.425523 18874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:38791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:38791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:22.426851 18874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:22.428138 18874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:22.439908 18879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:22.439880 18880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:22.439957 18882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:22.440922 18874 server_base.cc:1034] running on GCE node
I20260925 10:59:22.441730 18874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:22.443207 18874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:22.444422 18874 hybrid_clock.cc:648] HybridClock initialized: now 1790333962444365 us; error 42 us; skew 500 ppm
I20260925 10:59:22.444872 18874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:22 dist-test-slave-nx7s krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333962, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:22.448323 18874 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:59:22.450112 18874 webserver.cc:460] Webserver started at http://127.31.62.190:34241/ using document root <none> and password file <none>
I20260925 10:59:22.450857 18874 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:22.450985 18874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:22.451294 18874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:22.454628 18874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "b665f0383c414681870419ef4886eeb0"
format_stamp: "Formatted at 2026-09-25 10:59:22 on dist-test-slave-nx7s"
server_key: "7e105201818a54e5e8a229bf4cccf199"
server_key_iv: "05d6e70217f7bf23c86cd8d7a9ff65ef"
server_key_version: "encryptionkey@0"
I20260925 10:59:22.455451 18874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "b665f0383c414681870419ef4886eeb0"
format_stamp: "Formatted at 2026-09-25 10:59:22 on dist-test-slave-nx7s"
server_key: "7e105201818a54e5e8a229bf4cccf199"
server_key_iv: "05d6e70217f7bf23c86cd8d7a9ff65ef"
server_key_version: "encryptionkey@0"
I20260925 10:59:22.461292 18874 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 10:59:22.464993 18889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:22.466459 18874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:59:22.466624 18874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "b665f0383c414681870419ef4886eeb0"
format_stamp: "Formatted at 2026-09-25 10:59:22 on dist-test-slave-nx7s"
server_key: "7e105201818a54e5e8a229bf4cccf199"
server_key_iv: "05d6e70217f7bf23c86cd8d7a9ff65ef"
server_key_version: "encryptionkey@0"
I20260925 10:59:22.466813 18874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:22.486876 18874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:22.491291 18874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:22.491552 18874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:22.509404 18874 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:38791
I20260925 10:59:22.509457 18940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:38791 every 8 connection(s)
I20260925 10:59:22.510774 18874 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:22.514639 18941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:22.520327 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 18874
I20260925 10:59:22.520527 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:22.521095 31994 external_mini_cluster.cc:1428] Setting key 543a782baba07ecfc288039566e6dbb3
I20260925 10:59:22.526288 18941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0: Bootstrap starting.
I20260925 10:59:22.529973 18941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:22.531275 18941 log.cc:824] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:22.534932 18941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0: No bootstrap required, opened a new log
Sep 25 10:59:22 dist-test-slave-nx7s krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:59:22.543787 18941 raft_consensus.cc:348] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b665f0383c414681870419ef4886eeb0" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38791 } }
I20260925 10:59:22.544188 18941 raft_consensus.cc:374] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:22.544281 18941 raft_consensus.cc:729] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b665f0383c414681870419ef4886eeb0, State: Initialized, Role: FOLLOWER
I20260925 10:59:22.544916 18941 consensus_queue.cc:260] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b665f0383c414681870419ef4886eeb0" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38791 } }
I20260925 10:59:22.545188 18941 raft_consensus.cc:388] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:22.545198 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:22.523153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55944 (local address 127.31.62.190:38791)
0925 10:59:22.523998 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:22.524011 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:22.524055 (+ 44us) server_negotiation.cc:407] Connection header received
0925 10:59:22.525838 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:22.525863 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:22.526272 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:22.526635 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:22.528471 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:22.529879 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:22.531657 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:22.532723 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:22.532926 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:22.537654 (+ 4728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:22.537661 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:22.537680 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:22.537722 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:22.540691 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:22.541669 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:22.541676 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:22.541682 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:22.541857 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:22.542235 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:22.542238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:22.542240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:22.542831 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:22.543045 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:22.543522 (+ 477us) server_negotiation.cc:299] Negotiation successful
0925 10:59:22.543902 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":385,"threads_started":1}
I20260925 10:59:22.545331 18941 raft_consensus.cc:482] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:22.545522 18941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:22.547817 18941 raft_consensus.cc:504] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b665f0383c414681870419ef4886eeb0" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38791 } }
I20260925 10:59:22.548302 18941 leader_election.cc:302] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b665f0383c414681870419ef4886eeb0; no voters:
I20260925 10:59:22.548980 18941 leader_election.cc:288] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:22.549099 18946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:22.549700 18946 raft_consensus.cc:686] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [term 1 LEADER]: Becoming Leader. State: Replica: b665f0383c414681870419ef4886eeb0, State: Running, Role: LEADER
I20260925 10:59:22.550267 18946 consensus_queue.cc:237] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b665f0383c414681870419ef4886eeb0" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38791 } }
I20260925 10:59:22.550916 18941 sys_catalog.cc:564] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:22.552696 18947 sys_catalog.cc:455] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b665f0383c414681870419ef4886eeb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b665f0383c414681870419ef4886eeb0" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38791 } } }
I20260925 10:59:22.553148 18947 sys_catalog.cc:458] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:22.553884 18948 sys_catalog.cc:455] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b665f0383c414681870419ef4886eeb0. Latest consensus state: current_term: 1 leader_uuid: "b665f0383c414681870419ef4886eeb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b665f0383c414681870419ef4886eeb0" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38791 } } }
I20260925 10:59:22.554284 18948 sys_catalog.cc:458] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:22.561517 18951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:22.567476 18951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:22.577834 18951 catalog_manager.cc:1347] Generated new cluster ID: cfac4bc34a284ec989d7d97c62ab2a67
I20260925 10:59:22.577958 18951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:22.591085 18951 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:22.593026 18951 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:22.604194 18951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0: Generated new TSK 0
I20260925 10:59:22.605185 18951 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
Entry for principal kudu/127.31.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab.
I20260925 10:59:22.672721 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:38791
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:23.001313 18967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:23.001621 18967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:23.001744 18967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:23.009248 18967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:23.009367 18967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:23.009521 18967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:59:23.019583 18967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.129.keytab
--principal=kudu/127.31.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:38791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:23.020908 18967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:23.022243 18967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:23.035326 18973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:23.035386 18972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:23.035339 18975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:23.036540 18967 server_base.cc:1034] running on GCE node
I20260925 10:59:23.037257 18967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:23.038239 18967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:23.039448 18967 hybrid_clock.cc:648] HybridClock initialized: now 1790333963039381 us; error 59 us; skew 500 ppm
I20260925 10:59:23.039753 18967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:23 dist-test-slave-nx7s krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333963, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:23.042958 18967 init.cc:377] Logged in from keytab as kudu/127.31.62.129@KRBTEST.COM (short username kudu)
I20260925 10:59:23.044461 18967 webserver.cc:460] Webserver started at http://127.31.62.129:42953/ using document root <none> and password file <none>
I20260925 10:59:23.045389 18967 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:23.045542 18967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:23.045890 18967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:23.048902 18967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "87da07c92893467a96651ef003c3d9ed"
format_stamp: "Formatted at 2026-09-25 10:59:23 on dist-test-slave-nx7s"
server_key: "0a19165bf2fc07a8d7831beee16c0453"
server_key_iv: "79210fd3ccdd72e30282b51c6c599661"
server_key_version: "encryptionkey@0"
I20260925 10:59:23.049635 18967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "87da07c92893467a96651ef003c3d9ed"
format_stamp: "Formatted at 2026-09-25 10:59:23 on dist-test-slave-nx7s"
server_key: "0a19165bf2fc07a8d7831beee16c0453"
server_key_iv: "79210fd3ccdd72e30282b51c6c599661"
server_key_version: "encryptionkey@0"
I20260925 10:59:23.054849 18967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:59:23.058607 18982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:23.060070 18967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:59:23.060230 18967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "87da07c92893467a96651ef003c3d9ed"
format_stamp: "Formatted at 2026-09-25 10:59:23 on dist-test-slave-nx7s"
server_key: "0a19165bf2fc07a8d7831beee16c0453"
server_key_iv: "79210fd3ccdd72e30282b51c6c599661"
server_key_version: "encryptionkey@0"
I20260925 10:59:23.060444 18967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:23.081244 18967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:23.085216 18967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:23.085470 18967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:23.086416 18967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:23.088012 18967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:23.088090 18967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:23.088183 18967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:23.088225 18967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:23.118624 18967 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:42157
I20260925 10:59:23.118674 19094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:42157 every 8 connection(s)
I20260925 10:59:23.119951 18967 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:59:23.129818 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 18967
I20260925 10:59:23.130003 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:59:23.130618 31994 external_mini_cluster.cc:1428] Setting key 20333c71d8d62d82fda931c4cb462e79
Sep 25 10:59:23 dist-test-slave-nx7s krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333963, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.129@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:59:23.141113 19097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:23.122460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:55889 (local address 127.31.62.190:38791)
0925 10:59:23.122887 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:23.122892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:23.123898 (+ 1006us) server_negotiation.cc:407] Connection header received
0925 10:59:23.125853 (+ 1955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:23.125859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:23.125944 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:23.126043 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:23.127672 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:23.128398 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:23.130070 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:23.130763 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:23.130951 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:23.134850 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:23.134856 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:23.134862 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:23.134900 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:23.137834 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:23.138552 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:23.138557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:23.138561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:23.138705 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:23.139154 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:23.139160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:23.139163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:23.139544 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:23.139706 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:23.140444 (+ 738us) server_negotiation.cc:299] Negotiation successful
0925 10:59:23.140688 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":203,"threads_started":1}
I20260925 10:59:23.141811 19098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:23.123051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:38791 (local address 127.31.62.129:55889)
0925 10:59:23.123732 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:23.123771 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:23.125564 (+ 1793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:23.126311 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:23.126320 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:23.126858 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:23.127460 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:23.127478 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:23.128562 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:23.128566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:23.129883 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:23.129896 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:23.131025 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:23.131030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:23.131232 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:23.132072 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:59:23.132107 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:23.134628 (+ 2521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:23.137984 (+ 3356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:23.137992 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:23.138007 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:23.138433 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:23.138800 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:23.138803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:23.138806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:23.139001 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:23.139677 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:23.139683 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:23.140031 (+ 348us) client_negotiation.cc:769] Sending connection context
0925 10:59:23.140255 (+ 224us) client_negotiation.cc:240] Negotiation successful
0925 10:59:23.140672 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":250,"threads_started":1}
I20260925 10:59:23.143429 19095 heartbeater.cc:347] Connected to a master server at 127.31.62.190:38791
I20260925 10:59:23.143781 19095 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:23.144678 19095 heartbeater.cc:511] Master 127.31.62.190:38791 requested a full tablet report, sending...
I20260925 10:59:23.146910 18906 ts_manager.cc:194] Registered new tserver with Master: 87da07c92893467a96651ef003c3d9ed (127.31.62.129:42157)
I20260925 10:59:23.148417 18906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.129@KRBTEST.COM'} at 127.31.62.129:55889
WARNING: no policy specified for kudu/127.31.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
Entry for principal kudu/127.31.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab.
I20260925 10:59:23.187716 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:38791
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:23.520375 19101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:23.520646 19101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:23.520768 19101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:23.528156 19101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:23.528275 19101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:23.528429 19101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:59:23.538431 19101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.130.keytab
--principal=kudu/127.31.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:38791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:23.539707 19101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:23.541059 19101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:23.554004 19106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:23.553974 19109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:23.554716 19107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:23.554713 19101 server_base.cc:1034] running on GCE node
I20260925 10:59:23.555378 19101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:23.556391 19101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:23.557595 19101 hybrid_clock.cc:648] HybridClock initialized: now 1790333963557524 us; error 64 us; skew 500 ppm
I20260925 10:59:23.557958 19101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:23 dist-test-slave-nx7s krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333963, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:23.567176 19101 init.cc:377] Logged in from keytab as kudu/127.31.62.130@KRBTEST.COM (short username kudu)
I20260925 10:59:23.568797 19101 webserver.cc:460] Webserver started at http://127.31.62.130:39159/ using document root <none> and password file <none>
I20260925 10:59:23.569540 19101 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:23.569664 19101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:23.569983 19101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:23.573069 19101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "6aa66cbb166747e8a70e321f1789cc87"
format_stamp: "Formatted at 2026-09-25 10:59:23 on dist-test-slave-nx7s"
server_key: "161746e8df1f13e33f66efdc39b51ced"
server_key_iv: "865c127811d2a97401f80dc04a2bcb25"
server_key_version: "encryptionkey@0"
I20260925 10:59:23.573810 19101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "6aa66cbb166747e8a70e321f1789cc87"
format_stamp: "Formatted at 2026-09-25 10:59:23 on dist-test-slave-nx7s"
server_key: "161746e8df1f13e33f66efdc39b51ced"
server_key_iv: "865c127811d2a97401f80dc04a2bcb25"
server_key_version: "encryptionkey@0"
I20260925 10:59:23.579114 19101 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260925 10:59:23.582576 19116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:23.583962 19101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:59:23.584107 19101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "6aa66cbb166747e8a70e321f1789cc87"
format_stamp: "Formatted at 2026-09-25 10:59:23 on dist-test-slave-nx7s"
server_key: "161746e8df1f13e33f66efdc39b51ced"
server_key_iv: "865c127811d2a97401f80dc04a2bcb25"
server_key_version: "encryptionkey@0"
I20260925 10:59:23.584300 19101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:23.601853 19101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:23.607234 19101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:23.607529 19101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:23.608897 19101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:23.610510 19101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:23.610605 19101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:23.610746 19101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:23.610800 19101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:23.651203 19101 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:43991
I20260925 10:59:23.651224 19228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:43991 every 8 connection(s)
I20260925 10:59:23.652683 19101 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:59:23.662319 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 19101
I20260925 10:59:23.662511 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:59:23.662969 31994 external_mini_cluster.cc:1428] Setting key 3c3d6cc2f53539c9154cc5f6139f36c7
Sep 25 10:59:23 dist-test-slave-nx7s krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333963, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.130@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:59:23.673799 19231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:23.655210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:50079 (local address 127.31.62.190:38791)
0925 10:59:23.655696 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:23.655702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:23.656570 (+ 868us) server_negotiation.cc:407] Connection header received
0925 10:59:23.658635 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:23.658642 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:23.658732 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:23.658854 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:23.660495 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:23.661336 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:23.663127 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:23.663602 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:23.663737 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:23.667778 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:23.667783 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:23.667787 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:23.667825 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:23.670481 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:23.671309 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:23.671313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:23.671315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:23.671489 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:23.671951 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:23.671956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:23.671959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:23.672304 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:23.672458 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:23.673200 (+ 742us) server_negotiation.cc:299] Negotiation successful
0925 10:59:23.673426 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":231,"threads_started":1}
I20260925 10:59:23.674815 19232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:23.655671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:38791 (local address 127.31.62.130:50079)
0925 10:59:23.656401 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:23.656441 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:23.658332 (+ 1891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:23.659115 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:23.659125 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:23.659690 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:23.660308 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:23.660328 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:23.661522 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:23.661526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:23.662935 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:23.662949 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:23.663841 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:23.663849 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:23.664132 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:23.665175 (+ 1043us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:59:23.665211 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:23.667514 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:23.670650 (+ 3136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:23.670661 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:23.670682 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:23.671195 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:23.671594 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:23.671599 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:23.671603 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:23.671834 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:23.672434 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:23.672442 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:23.672783 (+ 341us) client_negotiation.cc:769] Sending connection context
0925 10:59:23.673107 (+ 324us) client_negotiation.cc:240] Negotiation successful
0925 10:59:23.673642 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":271,"threads_started":1}
I20260925 10:59:23.676303 19229 heartbeater.cc:347] Connected to a master server at 127.31.62.190:38791
I20260925 10:59:23.676613 19229 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:23.677363 19229 heartbeater.cc:511] Master 127.31.62.190:38791 requested a full tablet report, sending...
I20260925 10:59:23.678936 18906 ts_manager.cc:194] Registered new tserver with Master: 6aa66cbb166747e8a70e321f1789cc87 (127.31.62.130:43991)
I20260925 10:59:23.679757 18906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.130@KRBTEST.COM'} at 127.31.62.130:50079
WARNING: no policy specified for kudu/127.31.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
Entry for principal kudu/127.31.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab.
I20260925 10:59:23.715791 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:38791
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:24.039772 19235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:24.040071 19235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:24.040197 19235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:24.047760 19235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:24.047880 19235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:24.048039 19235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:59:24.057942 19235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.131.keytab
--principal=kudu/127.31.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:38791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:24.059221 19235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:24.060479 19235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:24.073487 19243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:24.073582 19240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:24.073555 19241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:24.075058 19235 server_base.cc:1034] running on GCE node
I20260925 10:59:24.075736 19235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:24.076720 19235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:24.077931 19235 hybrid_clock.cc:648] HybridClock initialized: now 1790333964077881 us; error 44 us; skew 500 ppm
I20260925 10:59:24.078246 19235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:24 dist-test-slave-nx7s krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333964, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:24.081390 19235 init.cc:377] Logged in from keytab as kudu/127.31.62.131@KRBTEST.COM (short username kudu)
I20260925 10:59:24.082863 19235 webserver.cc:460] Webserver started at http://127.31.62.131:35659/ using document root <none> and password file <none>
I20260925 10:59:24.083554 19235 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:24.083654 19235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:24.083911 19235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:24.087113 19235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "3161690a325d4eba989d87c117ec2204"
format_stamp: "Formatted at 2026-09-25 10:59:24 on dist-test-slave-nx7s"
server_key: "29569df591765c8ba8b6701fd50dd3bb"
server_key_iv: "b2ca9b27f0c32ef74b6aba0242e50285"
server_key_version: "encryptionkey@0"
I20260925 10:59:24.087904 19235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "3161690a325d4eba989d87c117ec2204"
format_stamp: "Formatted at 2026-09-25 10:59:24 on dist-test-slave-nx7s"
server_key: "29569df591765c8ba8b6701fd50dd3bb"
server_key_iv: "b2ca9b27f0c32ef74b6aba0242e50285"
server_key_version: "encryptionkey@0"
I20260925 10:59:24.093309 19235 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 10:59:24.096791 19250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:24.098266 19235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:59:24.098430 19235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "3161690a325d4eba989d87c117ec2204"
format_stamp: "Formatted at 2026-09-25 10:59:24 on dist-test-slave-nx7s"
server_key: "29569df591765c8ba8b6701fd50dd3bb"
server_key_iv: "b2ca9b27f0c32ef74b6aba0242e50285"
server_key_version: "encryptionkey@0"
I20260925 10:59:24.098631 19235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:24.113343 19235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:24.117321 19235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:24.117563 19235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:24.118628 19235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:24.120345 19235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:24.120476 19235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:24.120600 19235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:24.120702 19235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:24.150660 19235 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:36545
I20260925 10:59:24.150707 19362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:36545 every 8 connection(s)
I20260925 10:59:24.151968 19235 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:59:24.151968 19095 heartbeater.cc:503] Master 127.31.62.190:38791 was elected leader, sending a full tablet report...
I20260925 10:59:24.158967 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 19235
I20260925 10:59:24.159152 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:59:24.159557 31994 external_mini_cluster.cc:1428] Setting key 037cb7dfbb5c76a1829c5a35ff27f991
Sep 25 10:59:24 dist-test-slave-nx7s krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333964, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.131@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:59:24.172034 19231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.154529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:50563 (local address 127.31.62.190:38791)
0925 10:59:24.154737 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:24.154742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:24.156060 (+ 1318us) server_negotiation.cc:407] Connection header received
0925 10:59:24.158000 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:24.158004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:24.158080 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:24.158166 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:24.159868 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.160819 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.162688 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.163066 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.163177 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.166542 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:24.166547 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:24.166550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:24.166584 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:24.169090 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:24.169795 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:24.169799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:24.169801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:24.169918 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:24.170317 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:24.170322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:24.170325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:24.170606 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:24.170754 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:24.171472 (+ 718us) server_negotiation.cc:299] Negotiation successful
0925 10:59:24.171697 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 10:59:24.173089 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.155122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:38791 (local address 127.31.62.131:50563)
0925 10:59:24.155901 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:24.155939 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:24.157684 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:24.158434 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:24.158443 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:24.158991 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:24.159666 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:24.159691 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.160989 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.160993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:24.162559 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:24.162570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.163289 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.163297 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:24.163538 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.164147 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 10:59:24.164169 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:24.166340 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:24.169229 (+ 2889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:24.169239 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:24.169255 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:24.169677 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:24.170017 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 10:59:24.170022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 10:59:24.170026 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 10:59:24.170207 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 10:59:24.170730 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:24.170739 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:24.171091 (+ 352us) client_negotiation.cc:769] Sending connection context
0925 10:59:24.171360 (+ 269us) client_negotiation.cc:240] Negotiation successful
0925 10:59:24.171897 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":279,"threads_started":1}
I20260925 10:59:24.174629 19363 heartbeater.cc:347] Connected to a master server at 127.31.62.190:38791
I20260925 10:59:24.174983 19363 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:24.175793 19363 heartbeater.cc:511] Master 127.31.62.190:38791 requested a full tablet report, sending...
I20260925 10:59:24.177325 18906 ts_manager.cc:194] Registered new tserver with Master: 3161690a325d4eba989d87c117ec2204 (127.31.62.131:36545)
I20260925 10:59:24.178139 18906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.62.131@KRBTEST.COM'} at 127.31.62.131:50563
I20260925 10:59:24.187536 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:59:24.205051 19231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.192305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55950 (local address 127.31.62.190:38791)
0925 10:59:24.192497 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:24.192502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:24.192886 (+ 384us) server_negotiation.cc:407] Connection header received
0925 10:59:24.193099 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:24.193106 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:24.193195 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:24.193283 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:24.194723 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.195471 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.196796 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.197271 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.197369 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.199668 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:24.199672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:24.199675 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:24.199710 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:24.202334 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:24.203036 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:24.203040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:24.203044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:24.203200 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:24.203651 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:24.203654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:24.203656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:24.204003 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:24.204199 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:24.204547 (+ 348us) server_negotiation.cc:299] Negotiation successful
0925 10:59:24.204720 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 10:59:24.209151 18906 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:59:24.216112 18906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:59:24.251156 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.230993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:36545 (local address 127.0.0.1:57208)
0925 10:59:24.242795 (+ 11802us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:24.242810 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:24.242918 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:24.243520 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:24.243527 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:24.243557 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:24.243929 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:24.243940 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.246212 (+ 2272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.246218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:24.248313 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:24.248333 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.250075 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.250082 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:24.250284 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.250342 (+ 58us) client_negotiation.cc:769] Sending connection context
0925 10:59:24.250464 (+ 122us) client_negotiation.cc:240] Negotiation successful
0925 10:59:24.250665 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11689,"mutex_wait_us":200,"thread_start_us":11227,"threads_started":1}
I20260925 10:59:24.251147 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.231624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57208 (local address 127.31.62.131:36545)
0925 10:59:24.242347 (+ 10723us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:24.242356 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:24.242998 (+ 642us) server_negotiation.cc:407] Connection header received
0925 10:59:24.243109 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:24.243117 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:24.243325 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:24.243499 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:24.244124 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.245935 (+ 1811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.248477 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.249895 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.250016 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.250257 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:24.250536 (+ 279us) server_negotiation.cc:299] Negotiation successful
0925 10:59:24.250700 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10554,"thread_start_us":10391,"threads_started":1}
I20260925 10:59:24.252063 19375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.228726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:43991 (local address 127.0.0.1:54960)
0925 10:59:24.241395 (+ 12669us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:24.242751 (+ 1356us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:24.242984 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:24.243758 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:24.243762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:24.243788 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:24.244341 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:24.244349 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.246081 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.246086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:24.248826 (+ 2740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:24.248835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.251076 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.251083 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:24.251415 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.251441 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 10:59:24.251526 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 10:59:24.251659 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12334,"mutex_wait_us":231,"thread_start_us":12173,"threads_started":1}
I20260925 10:59:24.252090 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.228890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54960 (local address 127.31.62.130:43991)
0925 10:59:24.241822 (+ 12932us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:24.241831 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:24.242869 (+ 1038us) server_negotiation.cc:407] Connection header received
0925 10:59:24.243177 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:24.243183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:24.243345 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:24.243530 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:24.244555 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.245909 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.249025 (+ 3116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.250499 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.250610 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.250801 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:24.251604 (+ 803us) server_negotiation.cc:299] Negotiation successful
0925 10:59:24.251773 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12787,"thread_start_us":12655,"threads_started":1}
I20260925 10:59:24.252113 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.229156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41060 (local address 127.31.62.129:42157)
0925 10:59:24.240424 (+ 11268us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:24.240437 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:24.242578 (+ 2141us) server_negotiation.cc:407] Connection header received
0925 10:59:24.243066 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:24.243076 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:24.243347 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:24.243508 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:24.244522 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.246103 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.249026 (+ 2923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.250508 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.250610 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.250874 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:24.251604 (+ 730us) server_negotiation.cc:299] Negotiation successful
0925 10:59:24.251779 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11030,"thread_start_us":11311,"threads_started":1}
I20260925 10:59:24.252836 19379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.228883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:42157 (local address 127.0.0.1:41060)
0925 10:59:24.242346 (+ 13463us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:24.242413 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:24.242769 (+ 356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:24.243787 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:24.243791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:24.243812 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:24.244331 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:24.244343 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.246210 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.246214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:24.248819 (+ 2605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:24.248835 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.251082 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.251088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:24.251336 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.251427 (+ 91us) client_negotiation.cc:769] Sending connection context
0925 10:59:24.252361 (+ 934us) client_negotiation.cc:240] Negotiation successful
0925 10:59:24.252509 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13319,"mutex_wait_us":217,"thread_start_us":13654,"threads_started":1}
I20260925 10:59:24.255761 19164 tablet_service.cc:1467] Processing CreateTablet for tablet ac5d00ceacf4405295c394dd6798b763 (DEFAULT_TABLE table=test-table [id=63466273969b44a8b03474d3b2a04c9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:24.257038 19164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac5d00ceacf4405295c394dd6798b763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:24.256551 19298 tablet_service.cc:1467] Processing CreateTablet for tablet ac5d00ceacf4405295c394dd6798b763 (DEFAULT_TABLE table=test-table [id=63466273969b44a8b03474d3b2a04c9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:24.256551 19030 tablet_service.cc:1467] Processing CreateTablet for tablet ac5d00ceacf4405295c394dd6798b763 (DEFAULT_TABLE table=test-table [id=63466273969b44a8b03474d3b2a04c9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:24.257978 19298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac5d00ceacf4405295c394dd6798b763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:24.257982 19030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac5d00ceacf4405295c394dd6798b763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:24.268499 19381 tablet_bootstrap.cc:492] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87: Bootstrap starting.
I20260925 10:59:24.272210 19381 tablet_bootstrap.cc:654] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:24.273571 19382 tablet_bootstrap.cc:492] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204: Bootstrap starting.
I20260925 10:59:24.273583 19381 log.cc:824] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:24.275178 19383 tablet_bootstrap.cc:492] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed: Bootstrap starting.
I20260925 10:59:24.277661 19382 tablet_bootstrap.cc:654] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:24.278010 19383 tablet_bootstrap.cc:654] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:24.278945 19383 log.cc:824] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:24.279028 19382 log.cc:824] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:24.282187 19382 tablet_bootstrap.cc:492] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204: No bootstrap required, opened a new log
I20260925 10:59:24.282291 19381 tablet_bootstrap.cc:492] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87: No bootstrap required, opened a new log
I20260925 10:59:24.282461 19383 tablet_bootstrap.cc:492] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed: No bootstrap required, opened a new log
I20260925 10:59:24.282492 19382 ts_tablet_manager.cc:1397] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.004s
I20260925 10:59:24.282563 19381 ts_tablet_manager.cc:1397] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260925 10:59:24.282701 19383 ts_tablet_manager.cc:1397] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260925 10:59:24.290004 19381 raft_consensus.cc:348] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa66cbb166747e8a70e321f1789cc87" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 43991 } } peers { permanent_uuid: "87da07c92893467a96651ef003c3d9ed" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42157 } } peers { permanent_uuid: "3161690a325d4eba989d87c117ec2204" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36545 } }
I20260925 10:59:24.290414 19381 raft_consensus.cc:374] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:24.290510 19381 raft_consensus.cc:729] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aa66cbb166747e8a70e321f1789cc87, State: Initialized, Role: FOLLOWER
I20260925 10:59:24.291121 19381 consensus_queue.cc:260] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa66cbb166747e8a70e321f1789cc87" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 43991 } } peers { permanent_uuid: "87da07c92893467a96651ef003c3d9ed" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42157 } } peers { permanent_uuid: "3161690a325d4eba989d87c117ec2204" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36545 } }
I20260925 10:59:24.293186 19383 raft_consensus.cc:348] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa66cbb166747e8a70e321f1789cc87" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 43991 } } peers { permanent_uuid: "87da07c92893467a96651ef003c3d9ed" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42157 } } peers { permanent_uuid: "3161690a325d4eba989d87c117ec2204" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36545 } }
I20260925 10:59:24.293186 19382 raft_consensus.cc:348] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa66cbb166747e8a70e321f1789cc87" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 43991 } } peers { permanent_uuid: "87da07c92893467a96651ef003c3d9ed" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42157 } } peers { permanent_uuid: "3161690a325d4eba989d87c117ec2204" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36545 } }
I20260925 10:59:24.293650 19382 raft_consensus.cc:374] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:24.293650 19383 raft_consensus.cc:374] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:24.293756 19382 raft_consensus.cc:729] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3161690a325d4eba989d87c117ec2204, State: Initialized, Role: FOLLOWER
I20260925 10:59:24.293756 19383 raft_consensus.cc:729] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87da07c92893467a96651ef003c3d9ed, State: Initialized, Role: FOLLOWER
I20260925 10:59:24.294255 19229 heartbeater.cc:503] Master 127.31.62.190:38791 was elected leader, sending a full tablet report...
I20260925 10:59:24.294673 19381 ts_tablet_manager.cc:1428] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260925 10:59:24.294502 19383 consensus_queue.cc:260] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa66cbb166747e8a70e321f1789cc87" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 43991 } } peers { permanent_uuid: "87da07c92893467a96651ef003c3d9ed" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42157 } } peers { permanent_uuid: "3161690a325d4eba989d87c117ec2204" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36545 } }
I20260925 10:59:24.295164 19382 consensus_queue.cc:260] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa66cbb166747e8a70e321f1789cc87" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 43991 } } peers { permanent_uuid: "87da07c92893467a96651ef003c3d9ed" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42157 } } peers { permanent_uuid: "3161690a325d4eba989d87c117ec2204" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36545 } }
I20260925 10:59:24.296191 19383 ts_tablet_manager.cc:1428] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260925 10:59:24.296191 19363 heartbeater.cc:503] Master 127.31.62.190:38791 was elected leader, sending a full tablet report...
I20260925 10:59:24.297794 19382 ts_tablet_manager.cc:1428] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260925 10:59:24.372311 19096 tablet.cc:2369] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:59:24.403647 19364 tablet.cc:2369] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 10:59:24.404670 19230 tablet.cc:2369] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:59:24.406853 19388 raft_consensus.cc:482] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:59:24.407162 19388 raft_consensus.cc:504] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa66cbb166747e8a70e321f1789cc87" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 43991 } } peers { permanent_uuid: "87da07c92893467a96651ef003c3d9ed" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42157 } } peers { permanent_uuid: "3161690a325d4eba989d87c117ec2204" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36545 } }
I20260925 10:59:24.409099 19388 leader_election.cc:288] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6aa66cbb166747e8a70e321f1789cc87 (127.31.62.130:43991), 3161690a325d4eba989d87c117ec2204 (127.31.62.131:36545)
I20260925 10:59:24.418062 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.409561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:36323 (local address 127.31.62.131:36545)
0925 10:59:24.409798 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:24.409804 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:24.410549 (+ 745us) server_negotiation.cc:407] Connection header received
0925 10:59:24.410725 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:24.410730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:24.410808 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:24.410906 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:24.411670 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.412781 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.415695 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.416932 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.417093 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.417169 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:24.417626 (+ 457us) server_negotiation.cc:299] Negotiation successful
0925 10:59:24.417765 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260925 10:59:24.418062 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.409561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:50669 (local address 127.31.62.130:43991)
0925 10:59:24.409784 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:24.409792 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:24.410543 (+ 751us) server_negotiation.cc:407] Connection header received
0925 10:59:24.410724 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:24.410730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:24.410808 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:24.410906 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:24.411715 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.412779 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.415692 (+ 2913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.416922 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.417093 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.417169 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:24.417626 (+ 457us) server_negotiation.cc:299] Negotiation successful
0925 10:59:24.417765 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 10:59:24.418296 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.409323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:43991 (local address 127.31.62.129:50669)
0925 10:59:24.410410 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:24.410431 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:24.410607 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:24.410928 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:24.410933 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:24.410979 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:24.411584 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:24.411595 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.412947 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.412959 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:24.415507 (+ 2548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:24.415527 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.417171 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.417182 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:24.417439 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.417489 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 10:59:24.417581 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 10:59:24.417780 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"mutex_wait_us":235,"spinlock_wait_cycles":50688,"thread_start_us":712,"threads_started":1}
I20260925 10:59:24.418315 19391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.409444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:36545 (local address 127.31.62.129:36323)
0925 10:59:24.410454 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:24.410468 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:24.410596 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:24.410928 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:24.410933 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:24.410976 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:24.411487 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:24.411505 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.412948 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.412953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:24.415507 (+ 2554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:24.415527 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.417195 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.417200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:24.417445 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.417497 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 10:59:24.417581 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 10:59:24.417780 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"mutex_wait_us":208,"spinlock_wait_cycles":19584,"thread_start_us":616,"threads_started":1}
I20260925 10:59:24.419543 19318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac5d00ceacf4405295c394dd6798b763" candidate_uuid: "87da07c92893467a96651ef003c3d9ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3161690a325d4eba989d87c117ec2204" is_pre_election: true
I20260925 10:59:24.419541 19184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac5d00ceacf4405295c394dd6798b763" candidate_uuid: "87da07c92893467a96651ef003c3d9ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6aa66cbb166747e8a70e321f1789cc87" is_pre_election: true
I20260925 10:59:24.420113 19318 raft_consensus.cc:2454] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87da07c92893467a96651ef003c3d9ed in term 0.
I20260925 10:59:24.420113 19184 raft_consensus.cc:2454] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87da07c92893467a96651ef003c3d9ed in term 0.
I20260925 10:59:24.420912 18986 leader_election.cc:302] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3161690a325d4eba989d87c117ec2204, 6aa66cbb166747e8a70e321f1789cc87, 87da07c92893467a96651ef003c3d9ed; no voters:
I20260925 10:59:24.421399 19388 raft_consensus.cc:2781] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:59:24.421561 19388 raft_consensus.cc:482] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:59:24.421676 19388 raft_consensus.cc:3037] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:24.423821 19388 raft_consensus.cc:504] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa66cbb166747e8a70e321f1789cc87" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 43991 } } peers { permanent_uuid: "87da07c92893467a96651ef003c3d9ed" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42157 } } peers { permanent_uuid: "3161690a325d4eba989d87c117ec2204" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36545 } }
I20260925 10:59:24.424396 19388 leader_election.cc:288] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [CANDIDATE]: Term 1 election: Requested vote from peers 6aa66cbb166747e8a70e321f1789cc87 (127.31.62.130:43991), 3161690a325d4eba989d87c117ec2204 (127.31.62.131:36545)
I20260925 10:59:24.424993 19184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac5d00ceacf4405295c394dd6798b763" candidate_uuid: "87da07c92893467a96651ef003c3d9ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6aa66cbb166747e8a70e321f1789cc87"
I20260925 10:59:24.424993 19318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac5d00ceacf4405295c394dd6798b763" candidate_uuid: "87da07c92893467a96651ef003c3d9ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3161690a325d4eba989d87c117ec2204"
I20260925 10:59:24.425209 19318 raft_consensus.cc:3037] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:24.425209 19184 raft_consensus.cc:3037] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:24.428350 19318 raft_consensus.cc:2454] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87da07c92893467a96651ef003c3d9ed in term 1.
I20260925 10:59:24.428350 19184 raft_consensus.cc:2454] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87da07c92893467a96651ef003c3d9ed in term 1.
I20260925 10:59:24.428902 18984 leader_election.cc:302] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3161690a325d4eba989d87c117ec2204, 6aa66cbb166747e8a70e321f1789cc87, 87da07c92893467a96651ef003c3d9ed; no voters:
I20260925 10:59:24.429247 19388 raft_consensus.cc:2781] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:24.429672 19388 raft_consensus.cc:686] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [term 1 LEADER]: Becoming Leader. State: Replica: 87da07c92893467a96651ef003c3d9ed, State: Running, Role: LEADER
I20260925 10:59:24.430195 19388 consensus_queue.cc:237] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa66cbb166747e8a70e321f1789cc87" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 43991 } } peers { permanent_uuid: "87da07c92893467a96651ef003c3d9ed" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42157 } } peers { permanent_uuid: "3161690a325d4eba989d87c117ec2204" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36545 } }
I20260925 10:59:24.435060 18904 catalog_manager.cc:5495] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 87da07c92893467a96651ef003c3d9ed (127.31.62.129). New cstate: current_term: 1 leader_uuid: "87da07c92893467a96651ef003c3d9ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa66cbb166747e8a70e321f1789cc87" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 43991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87da07c92893467a96651ef003c3d9ed" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 42157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3161690a325d4eba989d87c117ec2204" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36545 } health_report { overall_health: UNKNOWN } } }
I20260925 10:59:24.491099 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.485896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41062 (local address 127.31.62.129:42157)
0925 10:59:24.486070 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:24.486074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:24.486091 (+ 17us) server_negotiation.cc:407] Connection header received
0925 10:59:24.486239 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:24.486244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:24.486324 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:24.486454 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:59:24.486767 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.487542 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.489089 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.489521 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.489611 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.490125 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:59:24.490641 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:59:24.490749 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:24.490826 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 10:59:24.490891 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 10:59:24.499272 19184 raft_consensus.cc:1261] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87 [term 1 FOLLOWER]: Refusing update from remote peer 87da07c92893467a96651ef003c3d9ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:59:24.499308 19318 raft_consensus.cc:1261] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204 [term 1 FOLLOWER]: Refusing update from remote peer 87da07c92893467a96651ef003c3d9ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:59:24.500217 19388 consensus_queue.cc:1035] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "6aa66cbb166747e8a70e321f1789cc87" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 43991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:59:24.500500 19392 consensus_queue.cc:1035] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3161690a325d4eba989d87c117ec2204" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:59:24.514348 19396 mvcc.cc:205] Tried to move back new op lower bound from 7333207918578270208 to 7333207918309748736. Current Snapshot: MvccSnapshot[applied={T|T < 7333207918578270208}]
I20260925 10:59:24.539443 18904 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:59:24.539716 18904 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:59:24.543112 18904 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b665f0383c414681870419ef4886eeb0: Sending DeleteTablet for 3 replicas of tablet ac5d00ceacf4405295c394dd6798b763
I20260925 10:59:24.544483 19030 tablet_service.cc:1514] Processing DeleteTablet for tablet ac5d00ceacf4405295c394dd6798b763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:24 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:41060
I20260925 10:59:24.544497 19298 tablet_service.cc:1514] Processing DeleteTablet for tablet ac5d00ceacf4405295c394dd6798b763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:24 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:57208
I20260925 10:59:24.545218 19164 tablet_service.cc:1514] Processing DeleteTablet for tablet ac5d00ceacf4405295c394dd6798b763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:24 UTC) from {username='kudu', principal='kudu/127.31.62.190@KRBTEST.COM'} at 127.0.0.1:54960
I20260925 10:59:24.545876 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 18967
I20260925 10:59:24.553388 19405 tablet_replica.cc:330] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed: stopping tablet replica
I20260925 10:59:24.553913 19405 raft_consensus.cc:2229] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:59:24.554630 19405 raft_consensus.cc:2258] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:24.557152 19406 tablet_replica.cc:330] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204: stopping tablet replica
I20260925 10:59:24.557595 19406 raft_consensus.cc:2229] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:59:24.558151 19406 raft_consensus.cc:2258] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:24.558948 19407 tablet_replica.cc:330] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87: stopping tablet replica
I20260925 10:59:24.558952 19405 ts_tablet_manager.cc:1905] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:24.559356 19407 raft_consensus.cc:2229] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:59:24.559952 19407 raft_consensus.cc:2258] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:24.562290 19407 ts_tablet_manager.cc:1905] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:24.565630 19406 ts_tablet_manager.cc:1905] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:24.566609 19405 ts_tablet_manager.cc:1918] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:24.566783 19405 log.cc:1196] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/ac5d00ceacf4405295c394dd6798b763
I20260925 10:59:24.567673 19405 ts_tablet_manager.cc:1939] T ac5d00ceacf4405295c394dd6798b763 P 87da07c92893467a96651ef003c3d9ed: Deleting consensus metadata
I20260925 10:59:24.569626 18892 catalog_manager.cc:4841] TS 87da07c92893467a96651ef003c3d9ed (127.31.62.129:42157): tablet ac5d00ceacf4405295c394dd6798b763 (table test-table [id=63466273969b44a8b03474d3b2a04c9c]) successfully deleted
Sep 25 10:59:24 dist-test-slave-nx7s krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.129@KRBTEST.COM
I20260925 10:59:24.572335 19406 ts_tablet_manager.cc:1918] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:24.572490 19406 log.cc:1196] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/ac5d00ceacf4405295c394dd6798b763
I20260925 10:59:24.572937 19406 ts_tablet_manager.cc:1939] T ac5d00ceacf4405295c394dd6798b763 P 3161690a325d4eba989d87c117ec2204: Deleting consensus metadata
I20260925 10:59:24.574617 18893 catalog_manager.cc:4841] TS 3161690a325d4eba989d87c117ec2204 (127.31.62.131:36545): tablet ac5d00ceacf4405295c394dd6798b763 (table test-table [id=63466273969b44a8b03474d3b2a04c9c]) successfully deleted
I20260925 10:59:24.578557 19407 ts_tablet_manager.cc:1918] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:24.578722 19407 log.cc:1196] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/ac5d00ceacf4405295c394dd6798b763
I20260925 10:59:24.579144 19407 ts_tablet_manager.cc:1939] T ac5d00ceacf4405295c394dd6798b763 P 6aa66cbb166747e8a70e321f1789cc87: Deleting consensus metadata
I20260925 10:59:24.580602 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.547094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.31.62.129:42157)
0925 10:59:24.547263 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:24.547269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:24.558343 (+ 11074us) server_negotiation.cc:407] Connection header received
0925 10:59:24.558910 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:24.558916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:24.559019 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:24.559125 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:24.561699 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.562937 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.565161 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.565900 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.566017 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.572176 (+ 6159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:24.572194 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:24.572214 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:24.572266 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:24.576896 (+ 4630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:24.577864 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:24.577874 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:24.577878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:24.578076 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:24.578563 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:24.578568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:24.578571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:24.579075 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:24.579369 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:24.579783 (+ 414us) server_negotiation.cc:299] Negotiation successful
0925 10:59:24.580050 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 10:59:24.581086 18891 catalog_manager.cc:4841] TS 6aa66cbb166747e8a70e321f1789cc87 (127.31.62.130:43991): tablet ac5d00ceacf4405295c394dd6798b763 (table test-table [id=63466273969b44a8b03474d3b2a04c9c]) successfully deleted
I20260925 10:59:24.581384 19090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:24.679622 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 18967
I20260925 10:59:24.694916 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 19101
Sep 25 10:59:24 dist-test-slave-nx7s krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.130@KRBTEST.COM
I20260925 10:59:24.709525 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.695707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54976 (local address 127.31.62.130:43991)
0925 10:59:24.695915 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:24.695920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:24.695942 (+ 22us) server_negotiation.cc:407] Connection header received
0925 10:59:24.696130 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:24.696135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:24.696201 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:24.696287 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:24.697592 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.698322 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.699583 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.699955 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.700050 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.704304 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:24.704322 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:24.704341 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:24.704377 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:24.707094 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:24.707739 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:24.707748 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:24.707750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:24.707927 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:24.708231 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:24.708235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:24.708237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:24.708633 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:24.708946 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:24.709098 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 10:59:24.709271 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260925 10:59:24.710117 19224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:24.798924 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 19101
I20260925 10:59:24.812871 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 19235
Sep 25 10:59:24 dist-test-slave-nx7s krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.131@KRBTEST.COM
I20260925 10:59:24.827646 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:24.813597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57212 (local address 127.31.62.131:36545)
0925 10:59:24.813747 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:24.813751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:24.813768 (+ 17us) server_negotiation.cc:407] Connection header received
0925 10:59:24.813922 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:24.813926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:24.813988 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:24.814069 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:24.815374 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.816196 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.817637 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:24.818095 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:24.818191 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:24.822342 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:24.822359 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:24.822376 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:24.822431 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:24.825084 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:24.825746 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:24.825754 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:24.825756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:24.825975 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:24.826321 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:24.826325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:24.826327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:24.826688 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:24.826927 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:24.827205 (+ 278us) server_negotiation.cc:299] Negotiation successful
0925 10:59:24.827387 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 10:59:24.828248 19358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:24.915860 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 19235
I20260925 10:59:24.929360 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 18874
I20260925 10:59:24.930339 18936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:25.039119 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 18874
2026-09-25T10:59:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4228 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-25T10:59:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:25Z Disabled control of system clock
I20260925 10:59:25.110074 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:35937
--webserver_interface=127.31.62.190
--webserver_port=0
--builtin_ntp_servers=127.31.62.148:37291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:35937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 10:59:25.435146 19416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:25.435446 19416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:25.435530 19416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:25.442848 19416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:25.442986 19416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:25.443022 19416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:25.443076 19416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:25.443118 19416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:25.452410 19416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:37291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:35937
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:35937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:25.453738 19416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:25.455467 19416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:25.466964 19424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:25.466912 19422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:25.466918 19421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:25.467744 19416 server_base.cc:1034] running on GCE node
I20260925 10:59:25.468551 19416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:25.470005 19416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:25.471191 19416 hybrid_clock.cc:648] HybridClock initialized: now 1790333965471134 us; error 51 us; skew 500 ppm
I20260925 10:59:25.471529 19416 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:59:25.478847 19416 webserver.cc:460] Webserver started at http://127.31.62.190:41315/ using document root <none> and password file <none>
I20260925 10:59:25.479580 19416 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:25.479676 19416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:25.480002 19416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:25.482939 19416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "8cd2f267510c4991beac7bbfd9e9c38f"
format_stamp: "Formatted at 2026-09-25 10:59:25 on dist-test-slave-nx7s"
server_key: "210a0c96f7dcb4e6ac011a9630f5a859"
server_key_iv: "3711622e6095e48acea0ad9e79dc935a"
server_key_version: "encryptionkey@0"
I20260925 10:59:25.483711 19416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "8cd2f267510c4991beac7bbfd9e9c38f"
format_stamp: "Formatted at 2026-09-25 10:59:25 on dist-test-slave-nx7s"
server_key: "210a0c96f7dcb4e6ac011a9630f5a859"
server_key_iv: "3711622e6095e48acea0ad9e79dc935a"
server_key_version: "encryptionkey@0"
I20260925 10:59:25.488739 19416 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260925 10:59:25.492235 19430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:25.493734 19416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:59:25.493877 19416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "8cd2f267510c4991beac7bbfd9e9c38f"
format_stamp: "Formatted at 2026-09-25 10:59:25 on dist-test-slave-nx7s"
server_key: "210a0c96f7dcb4e6ac011a9630f5a859"
server_key_iv: "3711622e6095e48acea0ad9e79dc935a"
server_key_version: "encryptionkey@0"
I20260925 10:59:25.494119 19416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:25.522105 19416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:25.522967 19416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:25.523236 19416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:25.541121 19416 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:35937
I20260925 10:59:25.541167 19481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:35937 every 8 connection(s)
I20260925 10:59:25.542431 19416 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:25.544580 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 19416
I20260925 10:59:25.544735 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:25.545167 31994 external_mini_cluster.cc:1428] Setting key 0b2026bcddf69ecc862b30bc1adf8273
I20260925 10:59:25.546579 19482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:25.557991 19485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:25.546985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45066 (local address 127.31.62.190:35937)
0925 10:59:25.548009 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:25.548020 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:25.548063 (+ 43us) server_negotiation.cc:407] Connection header received
0925 10:59:25.549652 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:25.549675 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:25.550176 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:25.550542 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:25.551095 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:25.552433 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:25.553765 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:25.554415 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:25.554580 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:25.555667 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:25.555677 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:25.555692 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:25.556274 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:25.556353 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:25.556377 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:25.556486 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:25.556701 (+ 215us) server_negotiation.cc:299] Negotiation successful
0925 10:59:25.557049 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":347,"threads_started":1}
I20260925 10:59:25.558584 19482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f: Bootstrap starting.
I20260925 10:59:25.561935 19482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:25.562889 19482 log.cc:824] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:25.565656 19482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f: No bootstrap required, opened a new log
I20260925 10:59:25.572830 19482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd2f267510c4991beac7bbfd9e9c38f" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35937 } }
I20260925 10:59:25.573259 19482 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:25.573354 19482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cd2f267510c4991beac7bbfd9e9c38f, State: Initialized, Role: FOLLOWER
I20260925 10:59:25.573957 19482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd2f267510c4991beac7bbfd9e9c38f" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35937 } }
I20260925 10:59:25.574175 19482 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:25.574276 19482 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:25.574411 19482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:25.576508 19482 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd2f267510c4991beac7bbfd9e9c38f" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35937 } }
I20260925 10:59:25.577039 19482 leader_election.cc:302] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cd2f267510c4991beac7bbfd9e9c38f; no voters:
I20260925 10:59:25.577579 19482 leader_election.cc:288] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:25.577728 19487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:25.578389 19487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [term 1 LEADER]: Becoming Leader. State: Replica: 8cd2f267510c4991beac7bbfd9e9c38f, State: Running, Role: LEADER
I20260925 10:59:25.578909 19487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd2f267510c4991beac7bbfd9e9c38f" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35937 } }
I20260925 10:59:25.579707 19482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:25.581904 19489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cd2f267510c4991beac7bbfd9e9c38f. Latest consensus state: current_term: 1 leader_uuid: "8cd2f267510c4991beac7bbfd9e9c38f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd2f267510c4991beac7bbfd9e9c38f" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35937 } } }
I20260925 10:59:25.582613 19489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:25.581972 19488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cd2f267510c4991beac7bbfd9e9c38f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd2f267510c4991beac7bbfd9e9c38f" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35937 } } }
I20260925 10:59:25.583305 19488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:25.584188 19496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:25.589505 19496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:25.598592 19496 catalog_manager.cc:1347] Generated new cluster ID: 1183b083690c4e9caf9315d2a4a08a62
I20260925 10:59:25.598711 19496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:25.620831 19496 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:25.622210 19496 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:25.630018 19496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f: Generated new TSK 0
I20260925 10:59:25.630873 19496 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:25.643970 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.129:0
--local_ip_for_outbound_sockets=127.31.62.129
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:35937
--builtin_ntp_servers=127.31.62.148:37291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 10:59:25.972040 19506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:25.972340 19506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:25.972476 19506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:25.979849 19506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:25.979959 19506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:25.980104 19506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.129
I20260925 10:59:25.989462 19506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:37291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.62.129
--webserver_port=0
--tserver_master_addrs=127.31.62.190:35937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.129
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:25.990846 19506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:25.992654 19506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:26.005452 19514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:26.005474 19512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:26.005496 19511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:26.006208 19506 server_base.cc:1034] running on GCE node
I20260925 10:59:26.006837 19506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:26.007844 19506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:26.009040 19506 hybrid_clock.cc:648] HybridClock initialized: now 1790333966008980 us; error 56 us; skew 500 ppm
I20260925 10:59:26.009343 19506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:59:26.011647 19506 webserver.cc:460] Webserver started at http://127.31.62.129:36083/ using document root <none> and password file <none>
I20260925 10:59:26.012310 19506 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:26.012393 19506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:26.012694 19506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:26.015671 19506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/data/instance:
uuid: "696fc5047ce24cc9a6bd0afbe765b260"
format_stamp: "Formatted at 2026-09-25 10:59:26 on dist-test-slave-nx7s"
server_key: "4eab9ad3c88ed1bccf3eb3aacfe78b1e"
server_key_iv: "02413cb5207c16262e6ca24174501c00"
server_key_version: "encryptionkey@0"
I20260925 10:59:26.016426 19506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance:
uuid: "696fc5047ce24cc9a6bd0afbe765b260"
format_stamp: "Formatted at 2026-09-25 10:59:26 on dist-test-slave-nx7s"
server_key: "4eab9ad3c88ed1bccf3eb3aacfe78b1e"
server_key_iv: "02413cb5207c16262e6ca24174501c00"
server_key_version: "encryptionkey@0"
I20260925 10:59:26.021791 19506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 10:59:26.025381 19520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:26.026749 19506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260925 10:59:26.026904 19506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/wal
uuid: "696fc5047ce24cc9a6bd0afbe765b260"
format_stamp: "Formatted at 2026-09-25 10:59:26 on dist-test-slave-nx7s"
server_key: "4eab9ad3c88ed1bccf3eb3aacfe78b1e"
server_key_iv: "02413cb5207c16262e6ca24174501c00"
server_key_version: "encryptionkey@0"
I20260925 10:59:26.027110 19506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:26.047369 19506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:26.048250 19506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:26.048573 19506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:26.049588 19506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:26.051093 19506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:26.051180 19506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:26.051290 19506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:26.051362 19506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:26.081446 19506 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.129:34599
I20260925 10:59:26.081498 19632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.129:34599 every 8 connection(s)
I20260925 10:59:26.082733 19506 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/data/info.pb
I20260925 10:59:26.091650 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 19506
I20260925 10:59:26.091828 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/wal/instance
I20260925 10:59:26.092352 31994 external_mini_cluster.cc:1428] Setting key 6481b0f9e2a4fb96e5149980e5cda134
I20260925 10:59:26.095455 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.130:0
--local_ip_for_outbound_sockets=127.31.62.130
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:35937
--builtin_ntp_servers=127.31.62.148:37291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 10:59:26.096382 19635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:26.085438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.129:43439 (local address 127.31.62.190:35937)
0925 10:59:26.085865 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:26.085870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:26.087086 (+ 1216us) server_negotiation.cc:407] Connection header received
0925 10:59:26.088982 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:26.088988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:26.089083 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:26.089176 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:26.090319 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:26.091009 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:26.093275 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:26.093859 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:26.094006 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:26.095513 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:26.095518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:26.095521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:26.095540 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:26.095588 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:26.095606 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:26.095695 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:26.095937 (+ 242us) server_negotiation.cc:299] Negotiation successful
0925 10:59:26.096055 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":224,"threads_started":1}
I20260925 10:59:26.097559 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:26.085982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:35937 (local address 127.31.62.129:43439)
0925 10:59:26.086911 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:26.086956 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:26.088674 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:26.089428 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:26.089436 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:26.089530 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:26.090140 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:26.090157 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:26.091163 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:26.091167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:26.093120 (+ 1953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:26.093135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:26.094087 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:26.094094 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:26.094387 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:26.095136 (+ 749us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:59:26.095140 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:26.095196 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:59:26.095313 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:26.095725 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:26.095736 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:26.095744 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 10:59:26.095910 (+ 166us) client_negotiation.cc:240] Negotiation successful
0925 10:59:26.096341 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":268,"threads_started":1}
I20260925 10:59:26.099012 19633 heartbeater.cc:347] Connected to a master server at 127.31.62.190:35937
I20260925 10:59:26.099347 19633 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:26.100342 19633 heartbeater.cc:511] Master 127.31.62.190:35937 requested a full tablet report, sending...
I20260925 10:59:26.102599 19447 ts_manager.cc:194] Registered new tserver with Master: 696fc5047ce24cc9a6bd0afbe765b260 (127.31.62.129:34599)
I20260925 10:59:26.104113 19447 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.62.129:43439
W20260925 10:59:26.423722 19637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:26.424036 19637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:26.424155 19637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:26.431911 19637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:26.432030 19637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:26.432178 19637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.130
I20260925 10:59:26.441665 19637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:37291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.62.130
--webserver_port=0
--tserver_master_addrs=127.31.62.190:35937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.130
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:26.443004 19637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:26.444866 19637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:26.457640 19642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:26.457656 19645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:26.457640 19643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:26.458717 19637 server_base.cc:1034] running on GCE node
I20260925 10:59:26.459378 19637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:26.460374 19637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:26.461591 19637 hybrid_clock.cc:648] HybridClock initialized: now 1790333966461520 us; error 59 us; skew 500 ppm
I20260925 10:59:26.461907 19637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:59:26.464570 19637 webserver.cc:460] Webserver started at http://127.31.62.130:33585/ using document root <none> and password file <none>
I20260925 10:59:26.465297 19637 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:26.465386 19637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:26.465657 19637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:26.468459 19637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/data/instance:
uuid: "de236671ab1141b8bedb425cb82f8192"
format_stamp: "Formatted at 2026-09-25 10:59:26 on dist-test-slave-nx7s"
server_key: "87c27561f3db08aabc58b172fe73e62b"
server_key_iv: "c8f39d1c0c16f13bac640b6a36f62c42"
server_key_version: "encryptionkey@0"
I20260925 10:59:26.469287 19637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance:
uuid: "de236671ab1141b8bedb425cb82f8192"
format_stamp: "Formatted at 2026-09-25 10:59:26 on dist-test-slave-nx7s"
server_key: "87c27561f3db08aabc58b172fe73e62b"
server_key_iv: "c8f39d1c0c16f13bac640b6a36f62c42"
server_key_version: "encryptionkey@0"
I20260925 10:59:26.474561 19637 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:59:26.478214 19651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:26.479671 19637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:59:26.479848 19637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/wal
uuid: "de236671ab1141b8bedb425cb82f8192"
format_stamp: "Formatted at 2026-09-25 10:59:26 on dist-test-slave-nx7s"
server_key: "87c27561f3db08aabc58b172fe73e62b"
server_key_iv: "c8f39d1c0c16f13bac640b6a36f62c42"
server_key_version: "encryptionkey@0"
I20260925 10:59:26.480094 19637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:26.494438 19637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:26.495262 19637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:26.495551 19637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:26.496490 19637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:26.498011 19637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:26.498108 19637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:26.498214 19637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:26.498281 19637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:26.528698 19637 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.130:38865
I20260925 10:59:26.528743 19763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.130:38865 every 8 connection(s)
I20260925 10:59:26.530028 19637 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/data/info.pb
I20260925 10:59:26.530514 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 19637
I20260925 10:59:26.530681 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/wal/instance
I20260925 10:59:26.531303 31994 external_mini_cluster.cc:1428] Setting key ade85f4bd9f1228096729b58d459cc01
I20260925 10:59:26.534128 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.62.131:0
--local_ip_for_outbound_sockets=127.31.62.131
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:35937
--builtin_ntp_servers=127.31.62.148:37291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 10:59:26.543553 19635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:26.532665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:56225 (local address 127.31.62.190:35937)
0925 10:59:26.532838 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:26.532868 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:26.534699 (+ 1831us) server_negotiation.cc:407] Connection header received
0925 10:59:26.536474 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:26.536480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:26.536560 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:26.536661 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:26.537752 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:26.538776 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:26.540336 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:26.540968 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:26.541086 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:26.542685 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:26.542690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:26.542693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:26.542714 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:26.542750 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:26.542763 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:26.542830 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:26.543127 (+ 297us) server_negotiation.cc:299] Negotiation successful
0925 10:59:26.543227 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 10:59:26.544775 19766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:26.533442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:35937 (local address 127.31.62.130:56225)
0925 10:59:26.534522 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:26.534569 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:26.536216 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:26.536867 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:26.536879 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:26.536987 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:26.537600 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:26.537618 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:26.538932 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:26.538936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:26.540185 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:26.540199 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:26.541221 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:26.541230 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:26.541530 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:26.542313 (+ 783us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:59:26.542317 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:26.542370 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:59:26.542481 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:26.542853 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:26.542863 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:26.542871 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 10:59:26.543074 (+ 203us) client_negotiation.cc:240] Negotiation successful
0925 10:59:26.543487 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":306,"threads_started":1}
I20260925 10:59:26.546334 19764 heartbeater.cc:347] Connected to a master server at 127.31.62.190:35937
I20260925 10:59:26.546672 19764 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:26.547600 19764 heartbeater.cc:511] Master 127.31.62.190:35937 requested a full tablet report, sending...
I20260925 10:59:26.549299 19447 ts_manager.cc:194] Registered new tserver with Master: de236671ab1141b8bedb425cb82f8192 (127.31.62.130:38865)
I20260925 10:59:26.550467 19447 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.62.130:56225
W20260925 10:59:26.860365 19767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:26.860670 19767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:26.860798 19767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:26.868160 19767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:26.868289 19767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:26.868422 19767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.62.131
I20260925 10:59:26.877910 19767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:37291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.62.131
--webserver_port=0
--tserver_master_addrs=127.31.62.190:35937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.62.131
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:26.879310 19767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:26.881500 19767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:26.894392 19775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:26.894443 19773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:26.894608 19772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:26.895393 19767 server_base.cc:1034] running on GCE node
I20260925 10:59:26.896039 19767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:26.897089 19767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:26.898296 19767 hybrid_clock.cc:648] HybridClock initialized: now 1790333966898246 us; error 42 us; skew 500 ppm
I20260925 10:59:26.898613 19767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 10:59:26.900990 19767 webserver.cc:460] Webserver started at http://127.31.62.131:33819/ using document root <none> and password file <none>
I20260925 10:59:26.901707 19767 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:26.901808 19767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:26.902107 19767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:26.905073 19767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/data/instance:
uuid: "2824590ab08f4d41bea74a8afcb21315"
format_stamp: "Formatted at 2026-09-25 10:59:26 on dist-test-slave-nx7s"
server_key: "b3b2e97510c427cb566e7158717c0fbe"
server_key_iv: "c2a59377064a845fc1620710e25756b9"
server_key_version: "encryptionkey@0"
I20260925 10:59:26.905860 19767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance:
uuid: "2824590ab08f4d41bea74a8afcb21315"
format_stamp: "Formatted at 2026-09-25 10:59:26 on dist-test-slave-nx7s"
server_key: "b3b2e97510c427cb566e7158717c0fbe"
server_key_iv: "c2a59377064a845fc1620710e25756b9"
server_key_version: "encryptionkey@0"
I20260925 10:59:26.911062 19767 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:59:26.914693 19781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:26.916131 19767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:59:26.916290 19767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/wal
uuid: "2824590ab08f4d41bea74a8afcb21315"
format_stamp: "Formatted at 2026-09-25 10:59:26 on dist-test-slave-nx7s"
server_key: "b3b2e97510c427cb566e7158717c0fbe"
server_key_iv: "c2a59377064a845fc1620710e25756b9"
server_key_version: "encryptionkey@0"
I20260925 10:59:26.916493 19767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:26.928956 19767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:26.929801 19767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:26.930091 19767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:26.931131 19767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 10:59:26.932627 19767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 10:59:26.932719 19767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:26.932827 19767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 10:59:26.932943 19767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260925 10:59:26.962844 19767 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.131:36179
I20260925 10:59:26.962913 19893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.131:36179 every 8 connection(s)
I20260925 10:59:26.964196 19767 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/data/info.pb
I20260925 10:59:26.972131 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 19767
I20260925 10:59:26.972295 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/wal/instance
I20260925 10:59:26.972738 31994 external_mini_cluster.cc:1428] Setting key 9998c35f3aee0de17c445b725b562594
I20260925 10:59:26.976755 19635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:26.966889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.131:44011 (local address 127.31.62.190:35937)
0925 10:59:26.967062 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:26.967066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:26.968474 (+ 1408us) server_negotiation.cc:407] Connection header received
0925 10:59:26.970279 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:26.970285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:26.970366 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:26.970465 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:26.971561 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:26.972279 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:26.974035 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:26.974544 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:26.974653 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:26.975988 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:26.975993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:26.975996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:26.976014 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:26.976045 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:26.976057 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:26.976126 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:26.976337 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 10:59:26.976434 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 10:59:26.977797 19896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:26.967442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:35937 (local address 127.31.62.131:44011)
0925 10:59:26.968308 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:26.968353 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:26.969998 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:26.970696 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:26.970706 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:26.970805 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0925 10:59:26.971390 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:26.971407 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:26.972435 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:26.972441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:26.973886 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:26.973901 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:26.974752 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:26.974760 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:26.974994 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:26.975664 (+ 670us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 10:59:26.975666 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 10:59:26.975714 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 10:59:26.975794 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 10:59:26.976140 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 10:59:26.976151 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 10:59:26.976159 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 10:59:26.976319 (+ 160us) client_negotiation.cc:240] Negotiation successful
0925 10:59:26.976727 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":256,"threads_started":1}
I20260925 10:59:26.979329 19894 heartbeater.cc:347] Connected to a master server at 127.31.62.190:35937
I20260925 10:59:26.979601 19894 heartbeater.cc:464] Registering TS with master...
I20260925 10:59:26.980365 19894 heartbeater.cc:511] Master 127.31.62.190:35937 requested a full tablet report, sending...
I20260925 10:59:26.981786 19447 ts_manager.cc:194] Registered new tserver with Master: 2824590ab08f4d41bea74a8afcb21315 (127.31.62.131:36179)
I20260925 10:59:26.982630 19447 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.62.131:44011
I20260925 10:59:26.988037 31994 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 10:59:26.997895 19635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:26.992084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33902 (local address 127.31.62.190:35937)
0925 10:59:26.992364 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:26.992371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:26.992648 (+ 277us) server_negotiation.cc:407] Connection header received
0925 10:59:26.992820 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:26.992825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:26.992948 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:26.993037 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:26.993538 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:26.994228 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:26.995604 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:26.995996 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:26.996107 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:26.997201 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:26.997224 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:26.997227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:26.997246 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:26.997279 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:26.997291 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:26.997360 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:26.997519 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 10:59:26.997599 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 10:59:27.001650 19447 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 10:59:27.008285 19447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 10:59:27.042680 19910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:27.020645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:36179 (local address 127.0.0.1:43410)
0925 10:59:27.034720 (+ 14075us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:27.034738 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:27.034870 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:27.035895 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:27.035902 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:27.035935 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:27.036388 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:27.036403 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.038282 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.038287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:27.040395 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:27.040407 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.041740 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.041750 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:27.041935 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:27.041998 (+ 63us) client_negotiation.cc:769] Sending connection context
0925 10:59:27.042107 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 10:59:27.042306 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13939,"mutex_wait_us":52,"thread_start_us":11793,"threads_started":1}
I20260925 10:59:27.043830 19909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:27.021242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43410 (local address 127.31.62.131:36179)
0925 10:59:27.033507 (+ 12265us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:27.033518 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:27.035005 (+ 1487us) server_negotiation.cc:407] Connection header received
0925 10:59:27.035121 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:27.035129 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:27.035417 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:27.035642 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:27.036624 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.038108 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.040563 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.041543 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.041662 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:27.041953 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:27.042174 (+ 221us) server_negotiation.cc:299] Negotiation successful
0925 10:59:27.042335 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12039,"thread_start_us":12277,"threads_started":1}
I20260925 10:59:27.044833 19907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:27.022956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34599 (local address 127.0.0.1:53298)
0925 10:59:27.036651 (+ 13695us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:27.036666 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:27.036783 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:27.037422 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:27.037427 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:27.037460 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:27.037791 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:27.037802 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.039563 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.039567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:27.041683 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:27.041696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.043849 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.043856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:27.044109 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:27.044168 (+ 59us) client_negotiation.cc:769] Sending connection context
0925 10:59:27.044250 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 10:59:27.044388 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13586,"mutex_wait_us":62,"thread_start_us":11491,"threads_started":1}
I20260925 10:59:27.044872 19911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:27.023461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53298 (local address 127.31.62.129:34599)
0925 10:59:27.034281 (+ 10820us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:27.034289 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:27.036783 (+ 2494us) server_negotiation.cc:407] Connection header received
0925 10:59:27.036963 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:27.036970 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:27.037205 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:27.037415 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:27.037953 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.039397 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.041903 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.043330 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.043459 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:27.043693 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:27.044330 (+ 637us) server_negotiation.cc:299] Negotiation successful
0925 10:59:27.044522 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10655,"thread_start_us":10721,"threads_started":1}
I20260925 10:59:27.045586 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:27.020712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58376 (local address 127.31.62.130:38865)
0925 10:59:27.036294 (+ 15582us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:27.036304 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:27.036337 (+ 33us) server_negotiation.cc:407] Connection header received
0925 10:59:27.036455 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:27.036464 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:27.036727 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:27.036970 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:27.037530 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.038970 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.041784 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.043330 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.043459 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:27.043723 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:27.045114 (+ 1391us) server_negotiation.cc:299] Negotiation successful
0925 10:59:27.045239 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15401,"thread_start_us":15298,"threads_started":1}
I20260925 10:59:27.046171 19906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:27.020603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.130:38865 (local address 127.0.0.1:58376)
0925 10:59:27.033005 (+ 12402us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:27.033073 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:27.033328 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:27.036970 (+ 3642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:27.036974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:27.036997 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:27.037326 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:27.037340 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.039131 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.039137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:27.041539 (+ 2402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:27.041554 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.043863 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.043869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:27.044104 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:27.044172 (+ 68us) client_negotiation.cc:769] Sending connection context
0925 10:59:27.044253 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 10:59:27.044388 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11998,"mutex_wait_us":98,"thread_start_us":15783,"threads_started":1}
I20260925 10:59:27.047840 19829 tablet_service.cc:1467] Processing CreateTablet for tablet c9b86607f4be484698240ffbc9e26dd9 (DEFAULT_TABLE table=test-table [id=80840af8279640ff91adcfa3e71b10d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:27.048604 19568 tablet_service.cc:1467] Processing CreateTablet for tablet c9b86607f4be484698240ffbc9e26dd9 (DEFAULT_TABLE table=test-table [id=80840af8279640ff91adcfa3e71b10d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:27.049441 19829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9b86607f4be484698240ffbc9e26dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:27.049480 19699 tablet_service.cc:1467] Processing CreateTablet for tablet c9b86607f4be484698240ffbc9e26dd9 (DEFAULT_TABLE table=test-table [id=80840af8279640ff91adcfa3e71b10d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 10:59:27.049969 19568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9b86607f4be484698240ffbc9e26dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:27.050730 19699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9b86607f4be484698240ffbc9e26dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:27.060657 19912 tablet_bootstrap.cc:492] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315: Bootstrap starting.
I20260925 10:59:27.063992 19913 tablet_bootstrap.cc:492] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192: Bootstrap starting.
I20260925 10:59:27.064836 19914 tablet_bootstrap.cc:492] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260: Bootstrap starting.
I20260925 10:59:27.066004 19912 tablet_bootstrap.cc:654] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:27.067391 19912 log.cc:824] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:27.068495 19914 tablet_bootstrap.cc:654] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:27.069741 19914 log.cc:824] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:27.070504 19913 tablet_bootstrap.cc:654] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:27.071273 19912 tablet_bootstrap.cc:492] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315: No bootstrap required, opened a new log
I20260925 10:59:27.071578 19912 ts_tablet_manager.cc:1397] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260925 10:59:27.071709 19913 log.cc:824] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:27.073110 19914 tablet_bootstrap.cc:492] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260: No bootstrap required, opened a new log
I20260925 10:59:27.073378 19914 ts_tablet_manager.cc:1397] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260925 10:59:27.074070 19913 tablet_bootstrap.cc:492] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192: No bootstrap required, opened a new log
I20260925 10:59:27.074290 19913 ts_tablet_manager.cc:1397] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260925 10:59:27.081224 19913 raft_consensus.cc:348] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de236671ab1141b8bedb425cb82f8192" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 38865 } } peers { permanent_uuid: "2824590ab08f4d41bea74a8afcb21315" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36179 } } peers { permanent_uuid: "696fc5047ce24cc9a6bd0afbe765b260" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34599 } }
I20260925 10:59:27.082054 19913 raft_consensus.cc:374] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:27.082206 19913 raft_consensus.cc:729] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de236671ab1141b8bedb425cb82f8192, State: Initialized, Role: FOLLOWER
I20260925 10:59:27.082854 19913 consensus_queue.cc:260] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de236671ab1141b8bedb425cb82f8192" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 38865 } } peers { permanent_uuid: "2824590ab08f4d41bea74a8afcb21315" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36179 } } peers { permanent_uuid: "696fc5047ce24cc9a6bd0afbe765b260" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34599 } }
I20260925 10:59:27.082758 19912 raft_consensus.cc:348] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de236671ab1141b8bedb425cb82f8192" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 38865 } } peers { permanent_uuid: "2824590ab08f4d41bea74a8afcb21315" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36179 } } peers { permanent_uuid: "696fc5047ce24cc9a6bd0afbe765b260" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34599 } }
I20260925 10:59:27.083221 19912 raft_consensus.cc:374] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:27.083431 19912 raft_consensus.cc:729] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2824590ab08f4d41bea74a8afcb21315, State: Initialized, Role: FOLLOWER
I20260925 10:59:27.084028 19764 heartbeater.cc:503] Master 127.31.62.190:35937 was elected leader, sending a full tablet report...
I20260925 10:59:27.084208 19912 consensus_queue.cc:260] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de236671ab1141b8bedb425cb82f8192" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 38865 } } peers { permanent_uuid: "2824590ab08f4d41bea74a8afcb21315" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36179 } } peers { permanent_uuid: "696fc5047ce24cc9a6bd0afbe765b260" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34599 } }
I20260925 10:59:27.084652 19913 ts_tablet_manager.cc:1428] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260925 10:59:27.086243 19894 heartbeater.cc:503] Master 127.31.62.190:35937 was elected leader, sending a full tablet report...
I20260925 10:59:27.086853 19912 ts_tablet_manager.cc:1428] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260925 10:59:27.087797 19914 raft_consensus.cc:348] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de236671ab1141b8bedb425cb82f8192" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 38865 } } peers { permanent_uuid: "2824590ab08f4d41bea74a8afcb21315" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36179 } } peers { permanent_uuid: "696fc5047ce24cc9a6bd0afbe765b260" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34599 } }
I20260925 10:59:27.088268 19914 raft_consensus.cc:374] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:27.088392 19914 raft_consensus.cc:729] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696fc5047ce24cc9a6bd0afbe765b260, State: Initialized, Role: FOLLOWER
I20260925 10:59:27.089039 19914 consensus_queue.cc:260] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de236671ab1141b8bedb425cb82f8192" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 38865 } } peers { permanent_uuid: "2824590ab08f4d41bea74a8afcb21315" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36179 } } peers { permanent_uuid: "696fc5047ce24cc9a6bd0afbe765b260" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34599 } }
I20260925 10:59:27.093585 19633 heartbeater.cc:503] Master 127.31.62.190:35937 was elected leader, sending a full tablet report...
I20260925 10:59:27.094235 19914 ts_tablet_manager.cc:1428] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260: Time spent starting tablet: real 0.021s user 0.008s sys 0.008s
I20260925 10:59:27.128880 19918 raft_consensus.cc:482] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:59:27.129202 19918 raft_consensus.cc:504] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de236671ab1141b8bedb425cb82f8192" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 38865 } } peers { permanent_uuid: "2824590ab08f4d41bea74a8afcb21315" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36179 } } peers { permanent_uuid: "696fc5047ce24cc9a6bd0afbe765b260" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34599 } }
I20260925 10:59:27.131126 19918 leader_election.cc:288] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2824590ab08f4d41bea74a8afcb21315 (127.31.62.131:36179), 696fc5047ce24cc9a6bd0afbe765b260 (127.31.62.129:34599)
I20260925 10:59:27.137255 19911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:27.132174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:33801 (local address 127.31.62.129:34599)
0925 10:59:27.132356 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:27.132361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:27.132382 (+ 21us) server_negotiation.cc:407] Connection header received
0925 10:59:27.132472 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:27.132477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:27.132550 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:27.132624 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:27.133222 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.133909 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.135648 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.136397 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.136544 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:27.136615 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:27.136888 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 10:59:27.137011 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 10:59:27.137343 19921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:27.131462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.129:34599 (local address 127.31.62.130:33801)
0925 10:59:27.132181 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:27.132201 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:27.132342 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:27.132664 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:27.132669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:27.132759 (+ 90us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:27.133076 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:27.133085 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.134039 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.134042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:27.135466 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:27.135478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.136580 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.136587 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:27.136720 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:27.136745 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 10:59:27.136826 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 10:59:27.136988 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"spinlock_wait_cycles":23424,"thread_start_us":344,"threads_started":1}
I20260925 10:59:27.137615 19909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:27.131692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.62.130:49069 (local address 127.31.62.131:36179)
0925 10:59:27.131984 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:27.131991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:27.132673 (+ 682us) server_negotiation.cc:407] Connection header received
0925 10:59:27.132815 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:27.132821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:27.132916 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:27.132992 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 10:59:27.133532 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.134344 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.136005 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.136785 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.136955 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:27.137020 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:27.137269 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 10:59:27.137400 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 10:59:27.137641 19922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:27.131529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.131:36179 (local address 127.31.62.130:49069)
0925 10:59:27.132564 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0925 10:59:27.132578 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 10:59:27.132702 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 10:59:27.133023 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 10:59:27.133031 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 10:59:27.133061 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 10:59:27.133387 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:27.133396 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.134469 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.134475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:27.135830 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 10:59:27.135842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.136968 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.136974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 10:59:27.137127 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:27.137150 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 10:59:27.137225 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 10:59:27.137353 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"thread_start_us":737,"threads_started":1}
I20260925 10:59:27.138449 19588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b86607f4be484698240ffbc9e26dd9" candidate_uuid: "de236671ab1141b8bedb425cb82f8192" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "696fc5047ce24cc9a6bd0afbe765b260" is_pre_election: true
I20260925 10:59:27.138935 19588 raft_consensus.cc:2454] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de236671ab1141b8bedb425cb82f8192 in term 0.
I20260925 10:59:27.138727 19849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b86607f4be484698240ffbc9e26dd9" candidate_uuid: "de236671ab1141b8bedb425cb82f8192" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2824590ab08f4d41bea74a8afcb21315" is_pre_election: true
I20260925 10:59:27.139407 19849 raft_consensus.cc:2454] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de236671ab1141b8bedb425cb82f8192 in term 0.
I20260925 10:59:27.139580 19653 leader_election.cc:302] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 696fc5047ce24cc9a6bd0afbe765b260, de236671ab1141b8bedb425cb82f8192; no voters:
I20260925 10:59:27.140000 19918 raft_consensus.cc:2781] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 10:59:27.140177 19918 raft_consensus.cc:482] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 10:59:27.140301 19918 raft_consensus.cc:3037] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:27.142696 19918 raft_consensus.cc:504] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de236671ab1141b8bedb425cb82f8192" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 38865 } } peers { permanent_uuid: "2824590ab08f4d41bea74a8afcb21315" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36179 } } peers { permanent_uuid: "696fc5047ce24cc9a6bd0afbe765b260" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34599 } }
I20260925 10:59:27.143375 19918 leader_election.cc:288] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [CANDIDATE]: Term 1 election: Requested vote from peers 2824590ab08f4d41bea74a8afcb21315 (127.31.62.131:36179), 696fc5047ce24cc9a6bd0afbe765b260 (127.31.62.129:34599)
I20260925 10:59:27.143846 19849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b86607f4be484698240ffbc9e26dd9" candidate_uuid: "de236671ab1141b8bedb425cb82f8192" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2824590ab08f4d41bea74a8afcb21315"
I20260925 10:59:27.144047 19849 raft_consensus.cc:3037] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:27.144030 19588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b86607f4be484698240ffbc9e26dd9" candidate_uuid: "de236671ab1141b8bedb425cb82f8192" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "696fc5047ce24cc9a6bd0afbe765b260"
I20260925 10:59:27.144227 19588 raft_consensus.cc:3037] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:27.146512 19849 raft_consensus.cc:2454] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de236671ab1141b8bedb425cb82f8192 in term 1.
I20260925 10:59:27.146553 19588 raft_consensus.cc:2454] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de236671ab1141b8bedb425cb82f8192 in term 1.
I20260925 10:59:27.146986 19654 leader_election.cc:302] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2824590ab08f4d41bea74a8afcb21315, de236671ab1141b8bedb425cb82f8192; no voters:
I20260925 10:59:27.147337 19918 raft_consensus.cc:2781] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:27.147831 19918 raft_consensus.cc:686] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [term 1 LEADER]: Becoming Leader. State: Replica: de236671ab1141b8bedb425cb82f8192, State: Running, Role: LEADER
I20260925 10:59:27.148334 19918 consensus_queue.cc:237] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de236671ab1141b8bedb425cb82f8192" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 38865 } } peers { permanent_uuid: "2824590ab08f4d41bea74a8afcb21315" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36179 } } peers { permanent_uuid: "696fc5047ce24cc9a6bd0afbe765b260" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34599 } }
I20260925 10:59:27.153292 19447 catalog_manager.cc:5495] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 reported cstate change: term changed from 0 to 1, leader changed from <none> to de236671ab1141b8bedb425cb82f8192 (127.31.62.130). New cstate: current_term: 1 leader_uuid: "de236671ab1141b8bedb425cb82f8192" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de236671ab1141b8bedb425cb82f8192" member_type: VOTER last_known_addr { host: "127.31.62.130" port: 38865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2824590ab08f4d41bea74a8afcb21315" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "696fc5047ce24cc9a6bd0afbe765b260" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34599 } health_report { overall_health: UNKNOWN } } }
I20260925 10:59:27.197012 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:27.191802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58384 (local address 127.31.62.130:38865)
0925 10:59:27.191978 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:27.191982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:27.192037 (+ 55us) server_negotiation.cc:407] Connection header received
0925 10:59:27.192134 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:27.192137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:27.192219 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:27.192359 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 10:59:27.192728 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.193543 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.194913 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.195323 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.195418 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:27.195925 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 10:59:27.196446 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 10:59:27.196591 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:27.196627 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 10:59:27.196690 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 10:59:27.209785 19849 raft_consensus.cc:1261] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315 [term 1 FOLLOWER]: Refusing update from remote peer de236671ab1141b8bedb425cb82f8192: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:59:27.210078 19588 raft_consensus.cc:1261] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260 [term 1 FOLLOWER]: Refusing update from remote peer de236671ab1141b8bedb425cb82f8192: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 10:59:27.210952 19923 consensus_queue.cc:1035] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [LEADER]: Connected to new peer: Peer: permanent_uuid: "696fc5047ce24cc9a6bd0afbe765b260" member_type: VOTER last_known_addr { host: "127.31.62.129" port: 34599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 10:59:27.211423 19918 consensus_queue.cc:1035] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2824590ab08f4d41bea74a8afcb21315" member_type: VOTER last_known_addr { host: "127.31.62.131" port: 36179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 10:59:27.215817 19895 tablet.cc:2369] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 10:59:27.244583 19446 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:59:27.244927 19446 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 10:59:27.249277 19446 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cd2f267510c4991beac7bbfd9e9c38f: Sending DeleteTablet for 3 replicas of tablet c9b86607f4be484698240ffbc9e26dd9
I20260925 10:59:27.250679 19568 tablet_service.cc:1514] Processing DeleteTablet for tablet c9b86607f4be484698240ffbc9e26dd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:27 UTC) from {username='slave'} at 127.0.0.1:53298
I20260925 10:59:27.250656 19699 tablet_service.cc:1514] Processing DeleteTablet for tablet c9b86607f4be484698240ffbc9e26dd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:27 UTC) from {username='slave'} at 127.0.0.1:58376
I20260925 10:59:27.251427 19829 tablet_service.cc:1514] Processing DeleteTablet for tablet c9b86607f4be484698240ffbc9e26dd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 10:59:27 UTC) from {username='slave'} at 127.0.0.1:43410
I20260925 10:59:27.251737 19937 tablet_replica.cc:330] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192: stopping tablet replica
I20260925 10:59:27.252112 19936 tablet_replica.cc:330] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260: stopping tablet replica
I20260925 10:59:27.252166 19937 raft_consensus.cc:2229] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [term 1 LEADER]: Raft consensus shutting down.
I20260925 10:59:27.252552 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 19506
I20260925 10:59:27.252594 19936 raft_consensus.cc:2229] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:59:27.252996 19937 raft_consensus.cc:2258] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:27.253235 19936 raft_consensus.cc:2258] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:27.255404 19937 ts_tablet_manager.cc:1905] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:27.255400 19936 ts_tablet_manager.cc:1905] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:27.262516 19937 ts_tablet_manager.cc:1918] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:27.262511 19936 ts_tablet_manager.cc:1918] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:27.262717 19936 log.cc:1196] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-0/wal/wals/c9b86607f4be484698240ffbc9e26dd9
I20260925 10:59:27.262850 19937 log.cc:1196] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-1/wal/wals/c9b86607f4be484698240ffbc9e26dd9
I20260925 10:59:27.263371 19936 ts_tablet_manager.cc:1939] T c9b86607f4be484698240ffbc9e26dd9 P 696fc5047ce24cc9a6bd0afbe765b260: Deleting consensus metadata
I20260925 10:59:27.263470 19937 ts_tablet_manager.cc:1939] T c9b86607f4be484698240ffbc9e26dd9 P de236671ab1141b8bedb425cb82f8192: Deleting consensus metadata
I20260925 10:59:27.265411 19432 catalog_manager.cc:4841] TS 696fc5047ce24cc9a6bd0afbe765b260 (127.31.62.129:34599): tablet c9b86607f4be484698240ffbc9e26dd9 (table test-table [id=80840af8279640ff91adcfa3e71b10d7]) successfully deleted
I20260925 10:59:27.265439 19434 catalog_manager.cc:4841] TS de236671ab1141b8bedb425cb82f8192 (127.31.62.130:38865): tablet c9b86607f4be484698240ffbc9e26dd9 (table test-table [id=80840af8279640ff91adcfa3e71b10d7]) successfully deleted
I20260925 10:59:27.266630 19938 tablet_replica.cc:330] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315: stopping tablet replica
I20260925 10:59:27.267067 19938 raft_consensus.cc:2229] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 10:59:27.267706 19938 raft_consensus.cc:2258] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 10:59:27.269892 19938 ts_tablet_manager.cc:1905] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 10:59:27.272977 19911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:27.255010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53300 (local address 127.31.62.129:34599)
0925 10:59:27.255197 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:27.255203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:27.265978 (+ 10775us) server_negotiation.cc:407] Connection header received
0925 10:59:27.266185 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:27.266191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:27.266300 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:27.266399 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:27.267007 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.268068 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.269865 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.270329 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.270445 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:27.271509 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:27.271517 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:27.271532 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:27.272174 (+ 642us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:27.272231 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:27.272253 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:27.272354 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:27.272495 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 10:59:27.272608 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 10:59:27.273592 19628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:27.277794 19938 ts_tablet_manager.cc:1918] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 10:59:27.277989 19938 log.cc:1196] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315: Deleting WAL directory at /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790333647382755-31994-0/minicluster-data/ts-2/wal/wals/c9b86607f4be484698240ffbc9e26dd9
I20260925 10:59:27.278542 19938 ts_tablet_manager.cc:1939] T c9b86607f4be484698240ffbc9e26dd9 P 2824590ab08f4d41bea74a8afcb21315: Deleting consensus metadata
I20260925 10:59:27.280468 19433 catalog_manager.cc:4841] TS 2824590ab08f4d41bea74a8afcb21315 (127.31.62.131:36179): tablet c9b86607f4be484698240ffbc9e26dd9 (table test-table [id=80840af8279640ff91adcfa3e71b10d7]) successfully deleted
I20260925 10:59:27.369984 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 19506
I20260925 10:59:27.384909 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 19637
I20260925 10:59:27.392799 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:27.385665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58398 (local address 127.31.62.130:38865)
0925 10:59:27.385943 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:27.385950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:27.385971 (+ 21us) server_negotiation.cc:407] Connection header received
0925 10:59:27.386056 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:27.386062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:27.386188 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:27.386301 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:27.386638 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.387845 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.389327 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.389977 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.390112 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:27.391173 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:27.391185 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:27.391205 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:27.392006 (+ 801us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:27.392084 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:27.392109 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:27.392211 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:27.392303 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 10:59:27.392409 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260925 10:59:27.393608 19759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:27.504092 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 19637
I20260925 10:59:27.518863 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 19767
I20260925 10:59:27.525437 19909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:27.519622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.31.62.131:36179)
0925 10:59:27.519794 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:27.519800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:27.519819 (+ 19us) server_negotiation.cc:407] Connection header received
0925 10:59:27.519949 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:27.519953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:27.520064 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:27.520150 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:27.520579 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.521370 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.522631 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:27.523030 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:27.523138 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:27.524122 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:27.524130 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:27.524145 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:27.524723 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:27.524771 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:27.524796 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:27.524928 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:27.525043 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 10:59:27.525148 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 10:59:27.526090 19889 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:27.627417 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 19767
I20260925 10:59:27.642148 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 19416
I20260925 10:59:27.643193 19477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:27.749960 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 19416
2026-09-25T10:59:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2713 ms)
[----------] 6 tests from EncryptionPolicyTest (21913 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:27 dist-test-slave-nx7s krb5kdc[19946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:27 dist-test-slave-nx7s krb5kdc[19946](info): set up 2 sockets
Sep 25 10:59:27 dist-test-slave-nx7s krb5kdc[19946](info): commencing operation
krb5kdc: starting...
W20260925 10:59:28.794895 31994 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:28 dist-test-slave-nx7s krb5kdc[19946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 10:59:29.044345 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37491
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:29.370343 19959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:29.370638 19959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:29.370784 19959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:29.378141 19959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:29.378265 19959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:29.378340 19959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:29.378417 19959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:29.378474 19959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:29.388464 19959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37491
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:29.389783 19959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:29.391230 19959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:29.403074 19967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:29.403079 19964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:29.403119 19965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:29.404300 19959 server_base.cc:1034] running on GCE node
I20260925 10:59:29.405278 19959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:29.406728 19959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:29.407956 19959 hybrid_clock.cc:648] HybridClock initialized: now 1790333969407899 us; error 49 us; skew 500 ppm
I20260925 10:59:29.408337 19959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:29 dist-test-slave-nx7s krb5kdc[19946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333969, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:29.411682 19959 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 10:59:29.413370 19959 webserver.cc:460] Webserver started at http://127.0.0.1:42247/ using document root <none> and password file <none>
I20260925 10:59:29.414161 19959 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:29.414284 19959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:29.414567 19959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:29.417912 19959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "23eb56fa2a4f44a386949c812d004b25"
format_stamp: "Formatted at 2026-09-25 10:59:29 on dist-test-slave-nx7s"
server_key: "a6acd3a2214fc8a9ab06dd6c16226a93"
server_key_iv: "c42507fe9e57780819e3d0302a86a900"
server_key_version: "encryptionkey@0"
I20260925 10:59:29.418804 19959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "23eb56fa2a4f44a386949c812d004b25"
format_stamp: "Formatted at 2026-09-25 10:59:29 on dist-test-slave-nx7s"
server_key: "a6acd3a2214fc8a9ab06dd6c16226a93"
server_key_iv: "c42507fe9e57780819e3d0302a86a900"
server_key_version: "encryptionkey@0"
I20260925 10:59:29.424415 19959 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 10:59:29.428010 19974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:29.429440 19959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260925 10:59:29.429617 19959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "23eb56fa2a4f44a386949c812d004b25"
format_stamp: "Formatted at 2026-09-25 10:59:29 on dist-test-slave-nx7s"
server_key: "a6acd3a2214fc8a9ab06dd6c16226a93"
server_key_iv: "c42507fe9e57780819e3d0302a86a900"
server_key_version: "encryptionkey@0"
I20260925 10:59:29.429809 19959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:29.458905 19959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:29.463017 19959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:29.463297 19959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:29.480860 19959 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37491
I20260925 10:59:29.480888 20025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37491 every 8 connection(s)
I20260925 10:59:29.482249 19959 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:29.486017 20026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:29.487502 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 19959
I20260925 10:59:29.487711 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:29.488514 31994 external_mini_cluster.cc:1428] Setting key 8c86f9880b65e283812cf7463c0840b9
I20260925 10:59:29.500053 20026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25: Bootstrap starting.
Sep 25 10:59:29 dist-test-slave-nx7s krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 10:59:29.504026 20026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:29.505261 20026 log.cc:824] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:29.508389 20026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25: No bootstrap required, opened a new log
I20260925 10:59:29.512082 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:29.490544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35401 (local address 127.0.0.1:37491)
0925 10:59:29.491341 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:29.491354 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:29.491404 (+ 50us) server_negotiation.cc:407] Connection header received
0925 10:59:29.493015 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:29.493040 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:29.493465 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:29.493794 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:29.495530 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:29.496958 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:29.498754 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:29.499484 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:29.499634 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:29.504468 (+ 4834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:29.504477 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:29.504496 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:29.504556 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:29.507602 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:29.508636 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:29.508643 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:29.508650 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:29.508819 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:29.509340 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:29.509344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:29.509346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:29.509886 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:29.510129 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:29.510658 (+ 529us) server_negotiation.cc:299] Negotiation successful
0925 10:59:29.510980 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":350,"threads_started":1}
I20260925 10:59:29.516346 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:29.517334 20026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23eb56fa2a4f44a386949c812d004b25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37491 } }
I20260925 10:59:29.517701 20026 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:29.517796 20026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23eb56fa2a4f44a386949c812d004b25, State: Initialized, Role: FOLLOWER
I20260925 10:59:29.518322 20026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23eb56fa2a4f44a386949c812d004b25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37491 } }
I20260925 10:59:29.518541 20026 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:29.518666 20026 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:29.518810 20026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:29.521157 20026 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23eb56fa2a4f44a386949c812d004b25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37491 } }
I20260925 10:59:29.521723 20026 leader_election.cc:302] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23eb56fa2a4f44a386949c812d004b25; no voters:
I20260925 10:59:29.522377 20026 leader_election.cc:288] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:29.522548 20036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:29.524462 20026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:29.525156 20036 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [term 1 LEADER]: Becoming Leader. State: Replica: 23eb56fa2a4f44a386949c812d004b25, State: Running, Role: LEADER
I20260925 10:59:29.525872 20036 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23eb56fa2a4f44a386949c812d004b25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37491 } }
I20260925 10:59:29.528792 20038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23eb56fa2a4f44a386949c812d004b25. Latest consensus state: current_term: 1 leader_uuid: "23eb56fa2a4f44a386949c812d004b25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23eb56fa2a4f44a386949c812d004b25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37491 } } }
I20260925 10:59:29.529299 20038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:29.529143 20037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23eb56fa2a4f44a386949c812d004b25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23eb56fa2a4f44a386949c812d004b25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37491 } } }
I20260925 10:59:29.529402 20037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:29.530105 20043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:29.539822 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:29.520998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57317 (local address 127.0.0.1:37491)
0925 10:59:29.521189 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:29.521195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:29.521576 (+ 381us) server_negotiation.cc:407] Connection header received
0925 10:59:29.521712 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:29.521718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:29.521800 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:29.521914 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:29.524047 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:29.525266 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:29.526958 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:29.527728 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:29.527868 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:29.531364 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:29.531371 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:29.531377 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:29.531422 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:29.535248 (+ 3826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:29.536366 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:29.536372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:29.536377 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:29.536551 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:29.537218 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:29.537223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:29.537227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:29.537720 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:29.537996 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:29.539187 (+ 1191us) server_negotiation.cc:299] Negotiation successful
0925 10:59:29.539409 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 10:59:29.549644 20043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:29.559158 20043 catalog_manager.cc:1347] Generated new cluster ID: af8a96c80559427db9e78bb77ecfc50f
I20260925 10:59:29.559284 20043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:29.566553 20043 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:29.567821 20043 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:29.578045 20043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23eb56fa2a4f44a386949c812d004b25: Generated new TSK 0
I20260925 10:59:29.579164 20043 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:29.594794 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 19959
I20260925 10:59:29.596191 20021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:29.687207 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 19959
W20260925 10:59:29.701097 19944 connection.cc:535] client connection to 127.0.0.1:37491 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37491): Connection reset by peer (error 104)
2026-09-25T10:59:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1929 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:29 dist-test-slave-nx7s krb5kdc[20058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:29 dist-test-slave-nx7s krb5kdc[20058](info): set up 2 sockets
Sep 25 10:59:29 dist-test-slave-nx7s krb5kdc[20058](info): commencing operation
krb5kdc: starting...
W20260925 10:59:30.705423 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:30 dist-test-slave-nx7s krb5kdc[20058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 10:59:30.951886 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35103
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:31.277446 20071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:31.277783 20071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:31.277927 20071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:31.285458 20071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:31.285602 20071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:31.285686 20071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:31.285773 20071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:31.285833 20071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:31.296293 20071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35103
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:31.297685 20071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:31.298998 20071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:31.310698 20076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:31.310742 20079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:31.310674 20077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:31.311510 20071 server_base.cc:1034] running on GCE node
I20260925 10:59:31.312294 20071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:31.313829 20071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:31.315048 20071 hybrid_clock.cc:648] HybridClock initialized: now 1790333971314989 us; error 49 us; skew 500 ppm
I20260925 10:59:31.315398 20071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:31 dist-test-slave-nx7s krb5kdc[20058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333971, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:31.322901 20071 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 10:59:31.324483 20071 webserver.cc:460] Webserver started at http://127.0.0.1:34465/ using document root <none> and password file <none>
I20260925 10:59:31.325253 20071 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:31.325382 20071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:31.325675 20071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:31.328725 20071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "6e642b2e0b894d998388750a2554b07a"
format_stamp: "Formatted at 2026-09-25 10:59:31 on dist-test-slave-nx7s"
server_key: "2a7884e353786e0eb7104a6f313999bd"
server_key_iv: "f2a88d2e8f8ae79e98c2171503f26136"
server_key_version: "encryptionkey@0"
I20260925 10:59:31.329559 20071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "6e642b2e0b894d998388750a2554b07a"
format_stamp: "Formatted at 2026-09-25 10:59:31 on dist-test-slave-nx7s"
server_key: "2a7884e353786e0eb7104a6f313999bd"
server_key_iv: "f2a88d2e8f8ae79e98c2171503f26136"
server_key_version: "encryptionkey@0"
I20260925 10:59:31.335041 20071 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:59:31.338755 20086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:31.340215 20071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 10:59:31.340386 20071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "6e642b2e0b894d998388750a2554b07a"
format_stamp: "Formatted at 2026-09-25 10:59:31 on dist-test-slave-nx7s"
server_key: "2a7884e353786e0eb7104a6f313999bd"
server_key_iv: "f2a88d2e8f8ae79e98c2171503f26136"
server_key_version: "encryptionkey@0"
I20260925 10:59:31.340574 20071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:31.357080 20071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:31.361955 20071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:31.362216 20071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:31.379397 20071 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35103
I20260925 10:59:31.379420 20137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35103 every 8 connection(s)
I20260925 10:59:31.380790 20071 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:31.384618 20138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:31.385717 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 20071
I20260925 10:59:31.385960 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:31.386381 31994 external_mini_cluster.cc:1428] Setting key 0052aec9795244249d3a60451b13b397
I20260925 10:59:31.397269 20138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a: Bootstrap starting.
I20260925 10:59:31.399343 20141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:31.388168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46633 (local address 127.0.0.1:35103)
0925 10:59:31.389075 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:31.389088 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:31.389134 (+ 46us) server_negotiation.cc:407] Connection header received
0925 10:59:31.390817 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:31.390841 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:31.391282 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:31.391672 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:31.392199 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:31.393538 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:31.394917 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:31.395623 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:31.395802 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:31.397037 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:31.397045 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:31.397060 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:31.397638 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:31.397730 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:31.397761 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:31.397899 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:31.398181 (+ 282us) server_negotiation.cc:299] Negotiation successful
0925 10:59:31.398347 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":334,"threads_started":1}
I20260925 10:59:31.400568 20138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:31.401966 20138 log.cc:824] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:31.403625 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:31.405040 20138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a: No bootstrap required, opened a new log
I20260925 10:59:31.414889 20141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:31.407944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55993 (local address 127.0.0.1:35103)
0925 10:59:31.408136 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:31.408142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:31.408418 (+ 276us) server_negotiation.cc:407] Connection header received
0925 10:59:31.408616 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:31.408620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:31.408691 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:31.408776 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:31.409363 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:31.410190 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:31.412047 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:31.412514 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:31.412610 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:31.413943 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:31.413949 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:31.413954 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:31.413981 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:31.414044 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:31.414065 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:31.414150 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:31.414448 (+ 298us) server_negotiation.cc:299] Negotiation successful
0925 10:59:31.414502 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 10:59:31.414613 20138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e642b2e0b894d998388750a2554b07a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } }
I20260925 10:59:31.415066 20138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:31.415189 20138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e642b2e0b894d998388750a2554b07a, State: Initialized, Role: FOLLOWER
I20260925 10:59:31.415891 20138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e642b2e0b894d998388750a2554b07a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } }
I20260925 10:59:31.416193 20138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:31.416340 20138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:31.416512 20138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:31.418769 20138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e642b2e0b894d998388750a2554b07a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } }
I20260925 10:59:31.419245 20138 leader_election.cc:302] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e642b2e0b894d998388750a2554b07a; no voters:
I20260925 10:59:31.419780 20138 leader_election.cc:288] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:31.419901 20148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:31.420547 20148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [term 1 LEADER]: Becoming Leader. State: Replica: 6e642b2e0b894d998388750a2554b07a, State: Running, Role: LEADER
I20260925 10:59:31.421101 20148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e642b2e0b894d998388750a2554b07a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } }
I20260925 10:59:31.422116 20138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:31.423786 20149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e642b2e0b894d998388750a2554b07a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e642b2e0b894d998388750a2554b07a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } } }
I20260925 10:59:31.424252 20149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:31.423797 20150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e642b2e0b894d998388750a2554b07a. Latest consensus state: current_term: 1 leader_uuid: "6e642b2e0b894d998388750a2554b07a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e642b2e0b894d998388750a2554b07a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } } }
I20260925 10:59:31.424657 20150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:31.425840 20153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:31.430481 20153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:31.439527 20153 catalog_manager.cc:1347] Generated new cluster ID: 1485e0cddaf44952932cbcf471b294b7
I20260925 10:59:31.439653 20153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:31.462971 20153 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:31.464329 20153 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:31.479154 20153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e642b2e0b894d998388750a2554b07a: Generated new TSK 0
I20260925 10:59:31.480118 20153 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:31.491945 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 20071
W20260925 10:59:31.493170 20133 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46633
W20260925 10:59:31.493865 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 20071: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:59:31.494031 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 20071
W20260925 10:59:31.507400 20056 connection.cc:535] client connection to 127.0.0.1:35103 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35103): Connection reset by peer (error 104)
2026-09-25T10:59:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1806 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:31 dist-test-slave-nx7s krb5kdc[20169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:31 dist-test-slave-nx7s krb5kdc[20169](info): set up 2 sockets
Sep 25 10:59:31 dist-test-slave-nx7s krb5kdc[20169](info): commencing operation
krb5kdc: starting...
W20260925 10:59:32.505537 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:32 dist-test-slave-nx7s krb5kdc[20169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 10:59:32.748204 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43453
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:33.076124 20182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:33.076435 20182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:33.076581 20182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:33.083871 20182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:33.084007 20182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:33.084087 20182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:33.084164 20182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:33.084224 20182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:33.094657 20182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43453
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:33.095968 20182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:33.097292 20182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:33.108451 20188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:33.108491 20190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:33.108520 20187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:33.109097 20182 server_base.cc:1034] running on GCE node
I20260925 10:59:33.109866 20182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:33.111220 20182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:33.112448 20182 hybrid_clock.cc:648] HybridClock initialized: now 1790333973112393 us; error 37 us; skew 500 ppm
I20260925 10:59:33.112808 20182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:33 dist-test-slave-nx7s krb5kdc[20169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:33.115926 20182 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 10:59:33.117393 20182 webserver.cc:460] Webserver started at http://127.0.0.1:45561/ using document root <none> and password file <none>
I20260925 10:59:33.118081 20182 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:33.118191 20182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:33.118480 20182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:33.121552 20182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "9a5983c5c68f406ca2464f6e3566c330"
format_stamp: "Formatted at 2026-09-25 10:59:33 on dist-test-slave-nx7s"
server_key: "9f6c0ce7e97a4137f6ef31bee0e58a57"
server_key_iv: "64e9ebf722e10ec348f75798517862ec"
server_key_version: "encryptionkey@0"
I20260925 10:59:33.122346 20182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "9a5983c5c68f406ca2464f6e3566c330"
format_stamp: "Formatted at 2026-09-25 10:59:33 on dist-test-slave-nx7s"
server_key: "9f6c0ce7e97a4137f6ef31bee0e58a57"
server_key_iv: "64e9ebf722e10ec348f75798517862ec"
server_key_version: "encryptionkey@0"
I20260925 10:59:33.127925 20182 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260925 10:59:33.131567 20197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:33.133010 20182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:59:33.133174 20182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "9a5983c5c68f406ca2464f6e3566c330"
format_stamp: "Formatted at 2026-09-25 10:59:33 on dist-test-slave-nx7s"
server_key: "9f6c0ce7e97a4137f6ef31bee0e58a57"
server_key_iv: "64e9ebf722e10ec348f75798517862ec"
server_key_version: "encryptionkey@0"
I20260925 10:59:33.133392 20182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:33.139133 20182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:33.143297 20182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:33.143584 20182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:33.161100 20182 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43453
I20260925 10:59:33.161139 20248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43453 every 8 connection(s)
I20260925 10:59:33.162479 20182 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:33.166215 20249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:33.171440 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 20182
I20260925 10:59:33.171638 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:33.172156 31994 external_mini_cluster.cc:1428] Setting key b54626cdc3506b1ddcc51b94cacfa07d
I20260925 10:59:33.178206 20249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330: Bootstrap starting.
I20260925 10:59:33.180213 20252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:33.174017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43097 (local address 127.0.0.1:43453)
0925 10:59:33.174807 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:33.174820 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:33.174866 (+ 46us) server_negotiation.cc:407] Connection header received
0925 10:59:33.176695 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:33.176728 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:33.177197 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:33.177564 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:33.178597 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:33.178604 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:33.178617 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:33.178641 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:33.178702 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:33.178730 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:33.178908 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:33.179020 (+ 112us) server_negotiation.cc:299] Negotiation successful
0925 10:59:33.179212 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":348,"threads_started":1}
I20260925 10:59:33.181717 20249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:33.182860 20249 log.cc:824] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:33.184149 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:33.185889 20249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330: No bootstrap required, opened a new log
I20260925 10:59:33.190997 20252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:33.188347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57189 (local address 127.0.0.1:43453)
0925 10:59:33.188566 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:33.188573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:33.189096 (+ 523us) server_negotiation.cc:407] Connection header received
0925 10:59:33.189272 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:33.189276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:33.189329 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:33.189414 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:33.190301 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:33.190305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:33.190308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:33.190324 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:33.190365 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:33.190383 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:33.190500 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:33.190615 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 10:59:33.190691 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260925 10:59:33.193771 20249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5983c5c68f406ca2464f6e3566c330" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43453 } }
I20260925 10:59:33.194164 20249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:33.194255 20249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a5983c5c68f406ca2464f6e3566c330, State: Initialized, Role: FOLLOWER
I20260925 10:59:33.194818 20249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5983c5c68f406ca2464f6e3566c330" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43453 } }
I20260925 10:59:33.195055 20249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:33.195180 20249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:33.195341 20249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:33.197872 20249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5983c5c68f406ca2464f6e3566c330" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43453 } }
I20260925 10:59:33.198378 20249 leader_election.cc:302] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a5983c5c68f406ca2464f6e3566c330; no voters:
I20260925 10:59:33.199013 20249 leader_election.cc:288] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:33.199154 20259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:33.199818 20259 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [term 1 LEADER]: Becoming Leader. State: Replica: 9a5983c5c68f406ca2464f6e3566c330, State: Running, Role: LEADER
I20260925 10:59:33.200518 20259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5983c5c68f406ca2464f6e3566c330" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43453 } }
I20260925 10:59:33.201195 20249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:33.203634 20260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a5983c5c68f406ca2464f6e3566c330" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5983c5c68f406ca2464f6e3566c330" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43453 } } }
I20260925 10:59:33.204010 20260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:33.203589 20261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a5983c5c68f406ca2464f6e3566c330. Latest consensus state: current_term: 1 leader_uuid: "9a5983c5c68f406ca2464f6e3566c330" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5983c5c68f406ca2464f6e3566c330" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43453 } } }
I20260925 10:59:33.204437 20261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:33.205073 20269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:33.210592 20269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:33.220254 20269 catalog_manager.cc:1347] Generated new cluster ID: 2268726085d34636867146a826c96232
I20260925 10:59:33.220381 20269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:33.241628 20269 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:33.242158 20269 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:33.251443 20269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a5983c5c68f406ca2464f6e3566c330: Generated new TSK 0
I20260925 10:59:33.252326 20269 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:33.260118 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 20182
W20260925 10:59:33.261469 20244 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43097
W20260925 10:59:33.262187 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 20182: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:59:33.262318 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 20182
2026-09-25T10:59:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1768 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:33 dist-test-slave-nx7s krb5kdc[20280](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:33 dist-test-slave-nx7s krb5kdc[20280](info): set up 2 sockets
Sep 25 10:59:33 dist-test-slave-nx7s krb5kdc[20280](info): commencing operation
krb5kdc: starting...
W20260925 10:59:34.297372 31994 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.002s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:34 dist-test-slave-nx7s krb5kdc[20280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 10:59:34.544397 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35249
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:34.867802 20293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:34.868141 20293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:34.868288 20293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:34.875645 20293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:34.875793 20293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:34.875873 20293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:34.875958 20293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:34.876014 20293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:34.886137 20293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:34.887513 20293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:34.888872 20293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:34.900563 20299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:34.900674 20298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:34.900579 20301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:34.901292 20293 server_base.cc:1034] running on GCE node
I20260925 10:59:34.902055 20293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:34.903450 20293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:34.904664 20293 hybrid_clock.cc:648] HybridClock initialized: now 1790333974904610 us; error 40 us; skew 500 ppm
I20260925 10:59:34.905043 20293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:34 dist-test-slave-nx7s krb5kdc[20280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333974, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:34.908105 20293 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 10:59:34.909613 20293 webserver.cc:460] Webserver started at http://127.0.0.1:43375/ using document root <none> and password file <none>
I20260925 10:59:34.910295 20293 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:34.910455 20293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:34.910758 20293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:34.913938 20293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "2ac2acf4af1549ffa2eddf7d9c2a705c"
format_stamp: "Formatted at 2026-09-25 10:59:34 on dist-test-slave-nx7s"
server_key: "1bb07ba7c4ebf492a7fa2d0582717aa2"
server_key_iv: "cbcbf0cef868afeac8d0a9d06f506f77"
server_key_version: "encryptionkey@0"
I20260925 10:59:34.914746 20293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "2ac2acf4af1549ffa2eddf7d9c2a705c"
format_stamp: "Formatted at 2026-09-25 10:59:34 on dist-test-slave-nx7s"
server_key: "1bb07ba7c4ebf492a7fa2d0582717aa2"
server_key_iv: "cbcbf0cef868afeac8d0a9d06f506f77"
server_key_version: "encryptionkey@0"
I20260925 10:59:34.920269 20293 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 10:59:34.924098 20308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:34.925591 20293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:59:34.925773 20293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "2ac2acf4af1549ffa2eddf7d9c2a705c"
format_stamp: "Formatted at 2026-09-25 10:59:34 on dist-test-slave-nx7s"
server_key: "1bb07ba7c4ebf492a7fa2d0582717aa2"
server_key_iv: "cbcbf0cef868afeac8d0a9d06f506f77"
server_key_version: "encryptionkey@0"
I20260925 10:59:34.925980 20293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:34.947799 20293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:34.952435 20293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:34.952706 20293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:34.970006 20293 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35249
I20260925 10:59:34.970064 20359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35249 every 8 connection(s)
I20260925 10:59:34.971426 20293 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:34.975221 20360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:34.977892 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 20293
I20260925 10:59:34.978156 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:34.978649 31994 external_mini_cluster.cc:1428] Setting key 319a518deec1deb88dd0072fa85b5088
I20260925 10:59:34.986568 20360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c: Bootstrap starting.
I20260925 10:59:34.989681 20360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:34.990695 20360 log.cc:824] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c: Log is configured to *not* fsync() on all Append() calls
Sep 25 10:59:34 dist-test-slave-nx7s krb5kdc[20280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 10:59:34.994027 20360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c: No bootstrap required, opened a new log
I20260925 10:59:35.001042 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:34.980468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38739 (local address 127.0.0.1:35249)
0925 10:59:34.981292 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:34.981304 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:34.981344 (+ 40us) server_negotiation.cc:407] Connection header received
0925 10:59:34.983075 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:34.983098 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:34.983540 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:34.983931 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:34.985439 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:34.986760 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:34.988010 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:34.988698 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:34.988885 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:34.993068 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:34.993077 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:34.993093 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:34.993135 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:34.996258 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:34.997185 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:34.997192 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:34.997199 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:34.997384 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:34.997781 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:34.997784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:34.997786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:34.998518 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:34.998831 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:34.999316 (+ 485us) server_negotiation.cc:299] Negotiation successful
0925 10:59:34.999800 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":381,"threads_started":1}
I20260925 10:59:35.003229 20360 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac2acf4af1549ffa2eddf7d9c2a705c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35249 } }
I20260925 10:59:35.003638 20360 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:35.003731 20360 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ac2acf4af1549ffa2eddf7d9c2a705c, State: Initialized, Role: FOLLOWER
I20260925 10:59:35.004347 20360 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac2acf4af1549ffa2eddf7d9c2a705c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35249 } }
I20260925 10:59:35.004598 20360 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:35.004742 20360 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:35.004899 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:35.004956 20360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:35.007164 20360 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac2acf4af1549ffa2eddf7d9c2a705c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35249 } }
I20260925 10:59:35.007683 20360 leader_election.cc:302] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ac2acf4af1549ffa2eddf7d9c2a705c; no voters:
I20260925 10:59:35.008355 20360 leader_election.cc:288] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:35.008533 20367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:35.009526 20367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [term 1 LEADER]: Becoming Leader. State: Replica: 2ac2acf4af1549ffa2eddf7d9c2a705c, State: Running, Role: LEADER
I20260925 10:59:35.010280 20367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac2acf4af1549ffa2eddf7d9c2a705c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35249 } }
I20260925 10:59:35.010812 20360 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:35.013244 20371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ac2acf4af1549ffa2eddf7d9c2a705c. Latest consensus state: current_term: 1 leader_uuid: "2ac2acf4af1549ffa2eddf7d9c2a705c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac2acf4af1549ffa2eddf7d9c2a705c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35249 } } }
I20260925 10:59:35.013698 20371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:35.014150 20370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ac2acf4af1549ffa2eddf7d9c2a705c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac2acf4af1549ffa2eddf7d9c2a705c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35249 } } }
I20260925 10:59:35.014611 20370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:35.015941 20377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:35.021072 20377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:35.033490 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:35.010010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36875 (local address 127.0.0.1:35249)
0925 10:59:35.010208 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:35.010214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:35.010939 (+ 725us) server_negotiation.cc:407] Connection header received
0925 10:59:35.011113 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:35.011117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:35.011199 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:35.011307 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:35.017161 (+ 5854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:35.018345 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:35.020357 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:35.021170 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:35.021274 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:35.025000 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:35.025008 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:35.025013 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:35.025070 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:35.029302 (+ 4232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:35.030627 (+ 1325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:35.030634 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:35.030638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:35.030813 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:35.031323 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:35.031332 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:35.031336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:35.031853 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:35.032108 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:35.032771 (+ 663us) server_negotiation.cc:299] Negotiation successful
0925 10:59:35.033062 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 10:59:35.038590 20377 catalog_manager.cc:1347] Generated new cluster ID: c6ff1d5e67f442fc8d30b6ca8c969daa
I20260925 10:59:35.039299 20377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:35.061050 20377 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:35.063033 20377 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:35.074033 20377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ac2acf4af1549ffa2eddf7d9c2a705c: Generated new TSK 0
I20260925 10:59:35.074901 20377 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:35.082273 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 20293
I20260925 10:59:35.083339 20355 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:35.175755 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 20293
2026-09-25T10:59:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1913 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:35 dist-test-slave-nx7s krb5kdc[20392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:35 dist-test-slave-nx7s krb5kdc[20392](info): set up 2 sockets
Sep 25 10:59:35 dist-test-slave-nx7s krb5kdc[20392](info): commencing operation
krb5kdc: starting...
W20260925 10:59:36.197415 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:36 dist-test-slave-nx7s krb5kdc[20392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 10:59:36.448324 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43063
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:36.774442 20405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:36.774748 20405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:36.774895 20405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:36.782330 20405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:36.782454 20405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:36.782526 20405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:36.782608 20405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:36.782667 20405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:36.792943 20405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:36.794302 20405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:36.795621 20405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:36.807271 20410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:36.807407 20411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:36.808065 20413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:36.808688 20405 server_base.cc:1034] running on GCE node
I20260925 10:59:36.809556 20405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:36.810993 20405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:36.812216 20405 hybrid_clock.cc:648] HybridClock initialized: now 1790333976812148 us; error 50 us; skew 500 ppm
I20260925 10:59:36.812587 20405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:36 dist-test-slave-nx7s krb5kdc[20392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333976, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:36.821024 20405 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 10:59:36.822726 20405 webserver.cc:460] Webserver started at http://127.0.0.1:34723/ using document root <none> and password file <none>
I20260925 10:59:36.823448 20405 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:36.823565 20405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:36.823863 20405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:36.827068 20405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "4fc3a6d08c1643969a7517f9ec835889"
format_stamp: "Formatted at 2026-09-25 10:59:36 on dist-test-slave-nx7s"
server_key: "bb8d02a6d7d1d1321a05d8c8c6cdd1ff"
server_key_iv: "4fd68b4d29c26d1623dcf4edf9b53333"
server_key_version: "encryptionkey@0"
I20260925 10:59:36.827858 20405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "4fc3a6d08c1643969a7517f9ec835889"
format_stamp: "Formatted at 2026-09-25 10:59:36 on dist-test-slave-nx7s"
server_key: "bb8d02a6d7d1d1321a05d8c8c6cdd1ff"
server_key_iv: "4fd68b4d29c26d1623dcf4edf9b53333"
server_key_version: "encryptionkey@0"
I20260925 10:59:36.833335 20405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 10:59:36.837038 20420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:36.838553 20405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:59:36.838747 20405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "4fc3a6d08c1643969a7517f9ec835889"
format_stamp: "Formatted at 2026-09-25 10:59:36 on dist-test-slave-nx7s"
server_key: "bb8d02a6d7d1d1321a05d8c8c6cdd1ff"
server_key_iv: "4fd68b4d29c26d1623dcf4edf9b53333"
server_key_version: "encryptionkey@0"
I20260925 10:59:36.838958 20405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:36.855247 20405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:36.859783 20405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:36.860064 20405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:36.877954 20405 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43063
I20260925 10:59:36.877983 20471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43063 every 8 connection(s)
I20260925 10:59:36.879316 20405 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:36.882750 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 20405
I20260925 10:59:36.882921 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:36.883355 20472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:36.883471 31994 external_mini_cluster.cc:1428] Setting key 91a7288cfdfbfb18302ff2e2ece7fbd5
I20260925 10:59:36.897229 20472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889: Bootstrap starting.
I20260925 10:59:36.897330 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:36.885209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40025 (local address 127.0.0.1:43063)
0925 10:59:36.885969 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:36.885980 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:36.886023 (+ 43us) server_negotiation.cc:407] Connection header received
0925 10:59:36.888006 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:36.888031 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:36.888457 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:36.888804 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:36.889405 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:36.890753 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:36.892479 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:36.893266 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:36.893410 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:36.894653 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:36.894662 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:36.894681 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:36.895370 (+ 689us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:36.895456 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:36.895494 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:36.895588 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:36.895894 (+ 306us) server_negotiation.cc:299] Negotiation successful
0925 10:59:36.896192 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":320,"threads_started":1}
I20260925 10:59:36.900540 20472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:36.901371 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:36.901815 20472 log.cc:824] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:36.905431 20472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889: No bootstrap required, opened a new log
I20260925 10:59:36.913528 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:36.906429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49229 (local address 127.0.0.1:43063)
0925 10:59:36.906614 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:36.906620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:36.907044 (+ 424us) server_negotiation.cc:407] Connection header received
0925 10:59:36.907205 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:36.907213 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:36.907299 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:36.907428 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:36.908102 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:36.909295 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:36.910731 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:36.911433 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:36.911577 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:36.912647 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:36.912653 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:36.912658 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:36.912689 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:36.912739 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:36.912763 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:36.912894 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:36.913051 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 10:59:36.913162 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 10:59:36.914906 20472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc3a6d08c1643969a7517f9ec835889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43063 } }
I20260925 10:59:36.915304 20472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:36.915410 20472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fc3a6d08c1643969a7517f9ec835889, State: Initialized, Role: FOLLOWER
I20260925 10:59:36.915992 20472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc3a6d08c1643969a7517f9ec835889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43063 } }
I20260925 10:59:36.916203 20472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:36.916332 20472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:36.916483 20472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:36.918838 20472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc3a6d08c1643969a7517f9ec835889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43063 } }
I20260925 10:59:36.919376 20472 leader_election.cc:302] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fc3a6d08c1643969a7517f9ec835889; no voters:
I20260925 10:59:36.920008 20472 leader_election.cc:288] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:36.920149 20482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:36.920910 20482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [term 1 LEADER]: Becoming Leader. State: Replica: 4fc3a6d08c1643969a7517f9ec835889, State: Running, Role: LEADER
I20260925 10:59:36.921499 20482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc3a6d08c1643969a7517f9ec835889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43063 } }
I20260925 10:59:36.922504 20472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:36.924572 20483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fc3a6d08c1643969a7517f9ec835889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc3a6d08c1643969a7517f9ec835889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43063 } } }
I20260925 10:59:36.924599 20484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fc3a6d08c1643969a7517f9ec835889. Latest consensus state: current_term: 1 leader_uuid: "4fc3a6d08c1643969a7517f9ec835889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc3a6d08c1643969a7517f9ec835889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43063 } } }
I20260925 10:59:36.925025 20483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:36.925118 20484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:36.926539 20488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:36.931752 20488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:36.941576 20488 catalog_manager.cc:1347] Generated new cluster ID: 32d5cf32e3c44d0688e681eac713a898
I20260925 10:59:36.941720 20488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:36.955562 20488 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:36.957484 20488 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:36.968338 20488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fc3a6d08c1643969a7517f9ec835889: Generated new TSK 0
I20260925 10:59:36.969296 20488 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:36.983115 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 20405
W20260925 10:59:36.984095 20467 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40025
W20260925 10:59:36.984779 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 20405: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:59:36.984949 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 20405
2026-09-25T10:59:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1808 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:37 dist-test-slave-nx7s krb5kdc[20503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:37 dist-test-slave-nx7s krb5kdc[20503](info): set up 2 sockets
Sep 25 10:59:37 dist-test-slave-nx7s krb5kdc[20503](info): commencing operation
krb5kdc: starting...
W20260925 10:59:37.997416 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:38 dist-test-slave-nx7s krb5kdc[20503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 10:59:38.240924 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46439
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:38.573881 20516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:38.574237 20516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:38.574380 20516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:38.581770 20516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:38.581893 20516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:38.581956 20516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:38.582046 20516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:38.582098 20516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:38.592451 20516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:38.593745 20516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:38.594995 20516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:38.606563 20522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:38.606652 20524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:38.606678 20521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:38.607702 20516 server_base.cc:1034] running on GCE node
I20260925 10:59:38.608526 20516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:38.610024 20516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:38.611231 20516 hybrid_clock.cc:648] HybridClock initialized: now 1790333978611176 us; error 49 us; skew 500 ppm
I20260925 10:59:38.611567 20516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:38 dist-test-slave-nx7s krb5kdc[20503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333978, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:38.614652 20516 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 10:59:38.616149 20516 webserver.cc:460] Webserver started at http://127.0.0.1:33693/ using document root <none> and password file <none>
I20260925 10:59:38.616806 20516 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:38.616959 20516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:38.617285 20516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:38.620278 20516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "84627ed3e25b4f678b6e73ec2e534ea1"
format_stamp: "Formatted at 2026-09-25 10:59:38 on dist-test-slave-nx7s"
server_key: "d1dbff672fd5b577745df59e7f302d5b"
server_key_iv: "2c4bd7685db7f717606c71ddc69f66b6"
server_key_version: "encryptionkey@0"
I20260925 10:59:38.621090 20516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "84627ed3e25b4f678b6e73ec2e534ea1"
format_stamp: "Formatted at 2026-09-25 10:59:38 on dist-test-slave-nx7s"
server_key: "d1dbff672fd5b577745df59e7f302d5b"
server_key_iv: "2c4bd7685db7f717606c71ddc69f66b6"
server_key_version: "encryptionkey@0"
I20260925 10:59:38.626489 20516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 10:59:38.630375 20531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:38.631789 20516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260925 10:59:38.631978 20516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "84627ed3e25b4f678b6e73ec2e534ea1"
format_stamp: "Formatted at 2026-09-25 10:59:38 on dist-test-slave-nx7s"
server_key: "d1dbff672fd5b577745df59e7f302d5b"
server_key_iv: "2c4bd7685db7f717606c71ddc69f66b6"
server_key_version: "encryptionkey@0"
I20260925 10:59:38.632169 20516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:38.638051 20516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:38.642009 20516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:38.642297 20516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:38.659578 20516 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46439
I20260925 10:59:38.659612 20582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46439 every 8 connection(s)
I20260925 10:59:38.660979 20516 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:38.664707 20583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:38.668327 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 20516
I20260925 10:59:38.668497 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:38.668936 31994 external_mini_cluster.cc:1428] Setting key fbf1d54d05ff9f5d5e77dfb4551a0771
I20260925 10:59:38.676267 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:38.670634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43831 (local address 127.0.0.1:46439)
0925 10:59:38.671382 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:38.671393 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:38.671434 (+ 41us) server_negotiation.cc:407] Connection header received
0925 10:59:38.673026 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:38.673048 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:38.673417 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:38.673760 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:38.674738 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:38.674744 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:38.674760 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:38.674773 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:38.674827 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:38.674848 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:38.675037 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:38.675137 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 10:59:38.675322 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":318,"threads_started":1}
I20260925 10:59:38.677187 20583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1: Bootstrap starting.
I20260925 10:59:38.679890 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:38.680673 20583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:38.681972 20583 log.cc:824] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:38.685653 20583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1: No bootstrap required, opened a new log
I20260925 10:59:38.687457 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:38.685203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55935 (local address 127.0.0.1:46439)
0925 10:59:38.685387 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:38.685393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:38.685644 (+ 251us) server_negotiation.cc:407] Connection header received
0925 10:59:38.685793 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:38.685798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:38.685866 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:38.685964 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:38.686796 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:38.686801 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:38.686813 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:38.686830 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:38.686881 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:38.686902 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:38.687011 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:38.687100 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 10:59:38.687158 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 10:59:38.693907 20583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84627ed3e25b4f678b6e73ec2e534ea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46439 } }
I20260925 10:59:38.694278 20583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:38.694383 20583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84627ed3e25b4f678b6e73ec2e534ea1, State: Initialized, Role: FOLLOWER
I20260925 10:59:38.695044 20583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84627ed3e25b4f678b6e73ec2e534ea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46439 } }
I20260925 10:59:38.695269 20583 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:38.695379 20583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:38.695503 20583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:38.697777 20583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84627ed3e25b4f678b6e73ec2e534ea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46439 } }
I20260925 10:59:38.698243 20583 leader_election.cc:302] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84627ed3e25b4f678b6e73ec2e534ea1; no voters:
I20260925 10:59:38.698783 20583 leader_election.cc:288] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:38.698918 20593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:38.699540 20593 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [term 1 LEADER]: Becoming Leader. State: Replica: 84627ed3e25b4f678b6e73ec2e534ea1, State: Running, Role: LEADER
I20260925 10:59:38.700071 20593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84627ed3e25b4f678b6e73ec2e534ea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46439 } }
I20260925 10:59:38.700723 20583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:38.703116 20594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84627ed3e25b4f678b6e73ec2e534ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84627ed3e25b4f678b6e73ec2e534ea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46439 } } }
I20260925 10:59:38.703547 20594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:38.703424 20595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84627ed3e25b4f678b6e73ec2e534ea1. Latest consensus state: current_term: 1 leader_uuid: "84627ed3e25b4f678b6e73ec2e534ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84627ed3e25b4f678b6e73ec2e534ea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46439 } } }
I20260925 10:59:38.703727 20595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:38.705711 20601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:38.710175 20601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:38.717993 20601 catalog_manager.cc:1347] Generated new cluster ID: a611d8f6d6e6411f945b1daa1a7529ee
I20260925 10:59:38.718128 20601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:38.758721 20601 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:38.759150 20601 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:38.765877 20601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84627ed3e25b4f678b6e73ec2e534ea1: Generated new TSK 0
I20260925 10:59:38.766675 20601 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:38.777073 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 20516
W20260925 10:59:38.778173 20578 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43831
W20260925 10:59:38.778857 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 20516: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:59:38.779031 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 20516
2026-09-25T10:59:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1794 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:38 dist-test-slave-nx7s krb5kdc[20614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:38 dist-test-slave-nx7s krb5kdc[20614](info): set up 2 sockets
Sep 25 10:59:38 dist-test-slave-nx7s krb5kdc[20614](info): commencing operation
krb5kdc: starting...
W20260925 10:59:39.801405 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:39 dist-test-slave-nx7s krb5kdc[20614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 10:59:40.046378 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44563
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:40.369215 20627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:40.369517 20627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:40.369650 20627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:40.376866 20627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:40.377013 20627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:40.377092 20627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:40.377174 20627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:40.377235 20627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:40.387266 20627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:40.388629 20627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:40.390043 20627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:40.401412 20632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:40.401484 20635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:40.401506 20633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:40.402491 20627 server_base.cc:1034] running on GCE node
I20260925 10:59:40.403244 20627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:40.404652 20627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:40.405848 20627 hybrid_clock.cc:648] HybridClock initialized: now 1790333980405787 us; error 56 us; skew 500 ppm
I20260925 10:59:40.406217 20627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:40 dist-test-slave-nx7s krb5kdc[20614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333980, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:40.409322 20627 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 10:59:40.410804 20627 webserver.cc:460] Webserver started at http://127.0.0.1:40767/ using document root <none> and password file <none>
I20260925 10:59:40.411468 20627 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:40.411567 20627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:40.411849 20627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:40.414965 20627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "5b875d3236a84bd492387942adbbfd80"
format_stamp: "Formatted at 2026-09-25 10:59:40 on dist-test-slave-nx7s"
server_key: "c684773cf6d3e761eeb376de8fd1a91b"
server_key_iv: "60a31d15b33a036d5ceb27e7378405fa"
server_key_version: "encryptionkey@0"
I20260925 10:59:40.415716 20627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "5b875d3236a84bd492387942adbbfd80"
format_stamp: "Formatted at 2026-09-25 10:59:40 on dist-test-slave-nx7s"
server_key: "c684773cf6d3e761eeb376de8fd1a91b"
server_key_iv: "60a31d15b33a036d5ceb27e7378405fa"
server_key_version: "encryptionkey@0"
I20260925 10:59:40.421121 20627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 10:59:40.424749 20642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:40.426357 20627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:59:40.426517 20627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "5b875d3236a84bd492387942adbbfd80"
format_stamp: "Formatted at 2026-09-25 10:59:40 on dist-test-slave-nx7s"
server_key: "c684773cf6d3e761eeb376de8fd1a91b"
server_key_iv: "60a31d15b33a036d5ceb27e7378405fa"
server_key_version: "encryptionkey@0"
I20260925 10:59:40.426702 20627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:40.448156 20627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:40.452513 20627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:40.452780 20627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:40.470258 20627 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44563
I20260925 10:59:40.470288 20693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44563 every 8 connection(s)
I20260925 10:59:40.471604 20627 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:40.475293 20694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:40.479832 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 20627
I20260925 10:59:40.479982 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:40.480367 31994 external_mini_cluster.cc:1428] Setting key ecae5d16dcf9cd4bc4995cf4a5fb8331
I20260925 10:59:40.486143 20694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80: Bootstrap starting.
I20260925 10:59:40.489811 20694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:40.490963 20694 log.cc:824] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:40.493875 20694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80: No bootstrap required, opened a new log
Sep 25 10:59:40 dist-test-slave-nx7s krb5kdc[20614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 10:59:40.501703 20694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b875d3236a84bd492387942adbbfd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44563 } }
I20260925 10:59:40.502070 20694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:40.502175 20694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b875d3236a84bd492387942adbbfd80, State: Initialized, Role: FOLLOWER
I20260925 10:59:40.502645 20697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:40.482208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:44503 (local address 127.0.0.1:44563)
0925 10:59:40.482965 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:40.482978 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:40.483022 (+ 44us) server_negotiation.cc:407] Connection header received
0925 10:59:40.484753 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:40.484779 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:40.485241 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:40.485558 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:40.487272 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:40.488593 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:40.490320 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:40.491040 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:40.491212 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:40.495378 (+ 4166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:40.495386 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:40.495404 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:40.495447 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:40.498456 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:40.499265 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:40.499272 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:40.499278 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:40.499450 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:40.499887 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:40.499893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:40.499895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:40.500442 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:40.500747 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:40.501224 (+ 477us) server_negotiation.cc:299] Negotiation successful
0925 10:59:40.501530 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":328,"threads_started":1}
I20260925 10:59:40.502785 20694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b875d3236a84bd492387942adbbfd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44563 } }
I20260925 10:59:40.503080 20694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:40.503213 20694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:40.503382 20694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:40.505973 20694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b875d3236a84bd492387942adbbfd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44563 } }
I20260925 10:59:40.506518 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:40.506503 20694 leader_election.cc:302] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b875d3236a84bd492387942adbbfd80; no voters:
I20260925 10:59:40.507136 20694 leader_election.cc:288] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:40.507244 20699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:40.509406 20694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:40.510751 20699 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [term 1 LEADER]: Becoming Leader. State: Replica: 5b875d3236a84bd492387942adbbfd80, State: Running, Role: LEADER
I20260925 10:59:40.511473 20699 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b875d3236a84bd492387942adbbfd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44563 } }
I20260925 10:59:40.519829 20701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b875d3236a84bd492387942adbbfd80. Latest consensus state: current_term: 1 leader_uuid: "5b875d3236a84bd492387942adbbfd80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b875d3236a84bd492387942adbbfd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44563 } } }
I20260925 10:59:40.520244 20701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:40.520828 20700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b875d3236a84bd492387942adbbfd80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b875d3236a84bd492387942adbbfd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44563 } } }
I20260925 10:59:40.521163 20700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:40.524156 20713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:40.526546 20697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:40.510955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:40509 (local address 127.0.0.1:44563)
0925 10:59:40.511115 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:40.511120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:40.511743 (+ 623us) server_negotiation.cc:407] Connection header received
0925 10:59:40.511991 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:40.511997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:40.512073 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:40.512150 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:40.513765 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:40.514565 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:40.515828 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:40.516273 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:40.516385 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:40.518790 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:40.518796 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:40.518803 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:40.518844 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:40.522737 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:40.523735 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:40.523741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:40.523744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:40.523908 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:40.524377 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:40.524382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:40.524386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:40.524893 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:40.525935 (+ 1042us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:40.526068 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 10:59:40.526200 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 10:59:40.530687 20713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:40.540145 20713 catalog_manager.cc:1347] Generated new cluster ID: 346af27860ba460b8238b396808550a2
I20260925 10:59:40.540262 20713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:40.560707 20713 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:40.562656 20713 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:40.572028 20713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b875d3236a84bd492387942adbbfd80: Generated new TSK 0
I20260925 10:59:40.572897 20713 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:40.586452 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 20627
I20260925 10:59:40.588606 20689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:40.680179 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 20627
W20260925 10:59:40.693856 20612 connection.cc:535] client connection to 127.0.0.1:44563 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44563): Connection reset by peer (error 104)
2026-09-25T10:59:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1901 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:40 dist-test-slave-nx7s krb5kdc[20726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:40 dist-test-slave-nx7s krb5kdc[20726](info): set up 2 sockets
Sep 25 10:59:40 dist-test-slave-nx7s krb5kdc[20726](info): commencing operation
krb5kdc: starting...
W20260925 10:59:41.693318 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:41 dist-test-slave-nx7s krb5kdc[20726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 10:59:41.940744 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37077
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37077
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:42.269129 20739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:42.269433 20739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:42.269531 20739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:42.276964 20739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:42.277091 20739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:42.277137 20739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:42.277195 20739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:42.277238 20739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:42.287405 20739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:42.288695 20739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:42.290033 20739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:42.301944 20747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:42.302026 20745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:42.302007 20744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:42.302999 20739 server_base.cc:1034] running on GCE node
I20260925 10:59:42.303897 20739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:42.305509 20739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:42.306742 20739 hybrid_clock.cc:648] HybridClock initialized: now 1790333982306684 us; error 56 us; skew 500 ppm
I20260925 10:59:42.307118 20739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:42 dist-test-slave-nx7s krb5kdc[20726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333982, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:42.315533 20739 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 10:59:42.317247 20739 webserver.cc:460] Webserver started at http://127.0.0.1:43081/ using document root <none> and password file <none>
I20260925 10:59:42.318046 20739 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:42.318190 20739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:42.318516 20739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:42.321743 20739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "e3f661c086e24f77899553977ac3a630"
format_stamp: "Formatted at 2026-09-25 10:59:42 on dist-test-slave-nx7s"
server_key: "6476e456aea2881bd0f3f8ae2bb86d4a"
server_key_iv: "3d5c1969be99a2149fca3b1d75b0b13d"
server_key_version: "encryptionkey@0"
I20260925 10:59:42.322551 20739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "e3f661c086e24f77899553977ac3a630"
format_stamp: "Formatted at 2026-09-25 10:59:42 on dist-test-slave-nx7s"
server_key: "6476e456aea2881bd0f3f8ae2bb86d4a"
server_key_iv: "3d5c1969be99a2149fca3b1d75b0b13d"
server_key_version: "encryptionkey@0"
I20260925 10:59:42.327883 20739 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 10:59:42.331457 20754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:42.332832 20739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:59:42.333036 20739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "e3f661c086e24f77899553977ac3a630"
format_stamp: "Formatted at 2026-09-25 10:59:42 on dist-test-slave-nx7s"
server_key: "6476e456aea2881bd0f3f8ae2bb86d4a"
server_key_iv: "3d5c1969be99a2149fca3b1d75b0b13d"
server_key_version: "encryptionkey@0"
I20260925 10:59:42.333226 20739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:42.363345 20739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:42.367666 20739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:42.367899 20739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:42.384919 20739 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37077
I20260925 10:59:42.384938 20805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37077 every 8 connection(s)
I20260925 10:59:42.386360 20739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:42.390272 20806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:42.394246 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 20739
I20260925 10:59:42.394411 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:42.394825 31994 external_mini_cluster.cc:1428] Setting key 4e5cce7c8488a231fad9d28401924760
I20260925 10:59:42.402561 20806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630: Bootstrap starting.
I20260925 10:59:42.405874 20806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:42.407071 20806 log.cc:824] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:42.407964 20809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:42.396669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:40955 (local address 127.0.0.1:37077)
0925 10:59:42.397448 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:42.397461 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:42.397503 (+ 42us) server_negotiation.cc:407] Connection header received
0925 10:59:42.399095 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:42.399120 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:42.399584 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:42.399979 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:42.400566 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:42.401944 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:42.403309 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:42.404024 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:42.404198 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:42.405499 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:42.405507 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:42.405522 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:42.406151 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:42.406242 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:42.406271 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:42.406420 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:42.406730 (+ 310us) server_negotiation.cc:299] Negotiation successful
0925 10:59:42.406898 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":326,"threads_started":1}
I20260925 10:59:42.410388 20806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630: No bootstrap required, opened a new log
I20260925 10:59:42.412393 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:42.418783 20806 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f661c086e24f77899553977ac3a630" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37077 } }
I20260925 10:59:42.419210 20806 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:42.419308 20806 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3f661c086e24f77899553977ac3a630, State: Initialized, Role: FOLLOWER
I20260925 10:59:42.420030 20806 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f661c086e24f77899553977ac3a630" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37077 } }
I20260925 10:59:42.420325 20806 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:42.420459 20806 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:42.420621 20806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:42.423617 20809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:42.416809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:55017 (local address 127.0.0.1:37077)
0925 10:59:42.417030 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:42.417036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:42.417366 (+ 330us) server_negotiation.cc:407] Connection header received
0925 10:59:42.417623 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:42.417628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:42.417703 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:42.417834 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:42.418494 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:42.419278 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:42.421164 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:42.421629 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:42.421736 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:42.422902 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:42.422907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:42.422910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:42.422948 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:42.423003 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:42.423020 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:42.423102 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:42.423262 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 10:59:42.423308 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260925 10:59:42.423574 20806 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f661c086e24f77899553977ac3a630" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37077 } }
I20260925 10:59:42.424117 20806 leader_election.cc:302] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3f661c086e24f77899553977ac3a630; no voters:
I20260925 10:59:42.424793 20806 leader_election.cc:288] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:42.424907 20816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:42.425601 20816 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [term 1 LEADER]: Becoming Leader. State: Replica: e3f661c086e24f77899553977ac3a630, State: Running, Role: LEADER
I20260925 10:59:42.426148 20816 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f661c086e24f77899553977ac3a630" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37077 } }
I20260925 10:59:42.427968 20816 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3f661c086e24f77899553977ac3a630. Latest consensus state: current_term: 1 leader_uuid: "e3f661c086e24f77899553977ac3a630" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f661c086e24f77899553977ac3a630" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37077 } } }
I20260925 10:59:42.428269 20816 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:42.428735 20818 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3f661c086e24f77899553977ac3a630" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f661c086e24f77899553977ac3a630" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37077 } } }
I20260925 10:59:42.429219 20818 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:42.432693 20806 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:42.432910 20820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:42.438885 20820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:42.448387 20820 catalog_manager.cc:1347] Generated new cluster ID: 65edbbdde5d14edd95b567e248cc7ea1
I20260925 10:59:42.448529 20820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:42.470553 20820 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:42.471935 20820 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:42.482748 20820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3f661c086e24f77899553977ac3a630: Generated new TSK 0
I20260925 10:59:42.483606 20820 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:42.492017 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 20739
W20260925 10:59:42.493674 20801 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.136:40955
W20260925 10:59:42.494431 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 20739: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:59:42.494570 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 20739
W20260925 10:59:42.507817 20724 connection.cc:535] client connection to 127.0.0.1:37077 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37077): Connection reset by peer (error 104)
2026-09-25T10:59:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1814 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:42 dist-test-slave-nx7s krb5kdc[20837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:42 dist-test-slave-nx7s krb5kdc[20837](info): set up 2 sockets
Sep 25 10:59:42 dist-test-slave-nx7s krb5kdc[20837](info): commencing operation
krb5kdc: starting...
W20260925 10:59:43.513382 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:43 dist-test-slave-nx7s krb5kdc[20837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 10:59:43.773124 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:44.101042 20850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:44.101336 20850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:44.101468 20850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:44.108755 20850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:44.108908 20850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:44.109002 20850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:44.109081 20850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:44.109143 20850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:44.119107 20850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:44.120455 20850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:44.121766 20850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:44.133507 20855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:44.133581 20858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:44.133558 20856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:44.134431 20850 server_base.cc:1034] running on GCE node
I20260925 10:59:44.135198 20850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:44.136673 20850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:44.137895 20850 hybrid_clock.cc:648] HybridClock initialized: now 1790333984137811 us; error 64 us; skew 500 ppm
I20260925 10:59:44.138266 20850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:44 dist-test-slave-nx7s krb5kdc[20837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333984, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:44.141366 20850 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 10:59:44.142902 20850 webserver.cc:460] Webserver started at http://127.0.0.1:38985/ using document root <none> and password file <none>
I20260925 10:59:44.143569 20850 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:44.143692 20850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:44.143985 20850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:44.147055 20850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "5f4a3a5826df4eb69f3a34dbd2da4cca"
format_stamp: "Formatted at 2026-09-25 10:59:44 on dist-test-slave-nx7s"
server_key: "0ca0c5667cc7cb159724a3aacf168df7"
server_key_iv: "19ad9f251b1b6f612629b01e8a79462d"
server_key_version: "encryptionkey@0"
I20260925 10:59:44.147866 20850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "5f4a3a5826df4eb69f3a34dbd2da4cca"
format_stamp: "Formatted at 2026-09-25 10:59:44 on dist-test-slave-nx7s"
server_key: "0ca0c5667cc7cb159724a3aacf168df7"
server_key_iv: "19ad9f251b1b6f612629b01e8a79462d"
server_key_version: "encryptionkey@0"
I20260925 10:59:44.153218 20850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260925 10:59:44.156939 20865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:44.158306 20850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:59:44.158514 20850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "5f4a3a5826df4eb69f3a34dbd2da4cca"
format_stamp: "Formatted at 2026-09-25 10:59:44 on dist-test-slave-nx7s"
server_key: "0ca0c5667cc7cb159724a3aacf168df7"
server_key_iv: "19ad9f251b1b6f612629b01e8a79462d"
server_key_version: "encryptionkey@0"
I20260925 10:59:44.158707 20850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:44.164394 20850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:44.168215 20850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:44.168488 20850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:44.185521 20850 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45695
I20260925 10:59:44.185557 20916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45695 every 8 connection(s)
I20260925 10:59:44.186990 20850 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:44.190682 20917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:44.196501 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 20850
I20260925 10:59:44.196660 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:44.197117 31994 external_mini_cluster.cc:1428] Setting key 268aef4c56ede13fbd0e8980e53ca7dd
I20260925 10:59:44.202435 20917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca: Bootstrap starting.
I20260925 10:59:44.206194 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:44.199049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:33825 (local address 127.0.0.1:45695)
0925 10:59:44.199908 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:44.199924 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:44.199977 (+ 53us) server_negotiation.cc:407] Connection header received
0925 10:59:44.202273 (+ 2296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:44.202303 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:44.202788 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:44.203226 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:44.204196 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:44.204206 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:44.204228 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:44.204246 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:44.204321 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:44.204348 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:44.204612 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:44.204737 (+ 125us) server_negotiation.cc:299] Negotiation successful
0925 10:59:44.204977 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":346,"threads_started":1}
I20260925 10:59:44.206501 20917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:44.207607 20917 log.cc:824] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:44.210201 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:44.210652 20917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca: No bootstrap required, opened a new log
I20260925 10:59:44.216893 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:44.214309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:52781 (local address 127.0.0.1:45695)
0925 10:59:44.214460 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:44.214464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:44.214923 (+ 459us) server_negotiation.cc:407] Connection header received
0925 10:59:44.215084 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:44.215090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:44.215143 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:44.215227 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:44.216158 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:44.216164 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:44.216169 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:44.216185 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:44.216228 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:44.216245 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:44.216369 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:44.216495 (+ 126us) server_negotiation.cc:299] Negotiation successful
0925 10:59:44.216560 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 10:59:44.218889 20917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4a3a5826df4eb69f3a34dbd2da4cca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45695 } }
I20260925 10:59:44.219259 20917 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:44.219355 20917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f4a3a5826df4eb69f3a34dbd2da4cca, State: Initialized, Role: FOLLOWER
I20260925 10:59:44.219960 20917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4a3a5826df4eb69f3a34dbd2da4cca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45695 } }
I20260925 10:59:44.220192 20917 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:44.220323 20917 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:44.220494 20917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:44.222728 20917 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4a3a5826df4eb69f3a34dbd2da4cca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45695 } }
I20260925 10:59:44.223296 20917 leader_election.cc:302] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f4a3a5826df4eb69f3a34dbd2da4cca; no voters:
I20260925 10:59:44.223948 20917 leader_election.cc:288] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:44.224066 20927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:44.224735 20927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [term 1 LEADER]: Becoming Leader. State: Replica: 5f4a3a5826df4eb69f3a34dbd2da4cca, State: Running, Role: LEADER
I20260925 10:59:44.225401 20927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4a3a5826df4eb69f3a34dbd2da4cca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45695 } }
I20260925 10:59:44.226400 20917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:44.228403 20928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f4a3a5826df4eb69f3a34dbd2da4cca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4a3a5826df4eb69f3a34dbd2da4cca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45695 } } }
I20260925 10:59:44.228946 20928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:44.229866 20936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:44.229691 20929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f4a3a5826df4eb69f3a34dbd2da4cca. Latest consensus state: current_term: 1 leader_uuid: "5f4a3a5826df4eb69f3a34dbd2da4cca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4a3a5826df4eb69f3a34dbd2da4cca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45695 } } }
I20260925 10:59:44.230188 20929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:44.236059 20936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:44.245816 20936 catalog_manager.cc:1347] Generated new cluster ID: 8f5f27cc0931403fbf71a507393e3edc
I20260925 10:59:44.245956 20936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:44.256996 20936 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:44.257454 20936 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:44.268783 20936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f4a3a5826df4eb69f3a34dbd2da4cca: Generated new TSK 0
I20260925 10:59:44.269843 20936 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:44.283977 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 20850
W20260925 10:59:44.285112 20912 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.136:33825
W20260925 10:59:44.285847 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 20850: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:59:44.286000 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 20850
2026-09-25T10:59:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1791 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:44 dist-test-slave-nx7s krb5kdc[20948](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:44 dist-test-slave-nx7s krb5kdc[20948](info): set up 2 sockets
Sep 25 10:59:44 dist-test-slave-nx7s krb5kdc[20948](info): commencing operation
krb5kdc: starting...
W20260925 10:59:45.305642 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.988s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:45 dist-test-slave-nx7s krb5kdc[20948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 10:59:45.552403 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36053
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:45.879035 20961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:45.879334 20961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:45.879441 20961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:45.886919 20961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:45.887051 20961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:45.887099 20961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:45.887157 20961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:45.887202 20961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:45.897755 20961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36053
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:45.899053 20961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:45.900354 20961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:45.911751 20966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:45.911804 20967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:45.911772 20969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:45.912648 20961 server_base.cc:1034] running on GCE node
I20260925 10:59:45.913491 20961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:45.915014 20961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:45.916201 20961 hybrid_clock.cc:648] HybridClock initialized: now 1790333985916149 us; error 44 us; skew 500 ppm
I20260925 10:59:45.916548 20961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:45 dist-test-slave-nx7s krb5kdc[20948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333985, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:45.919654 20961 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 10:59:45.921239 20961 webserver.cc:460] Webserver started at http://127.0.0.1:46661/ using document root <none> and password file <none>
I20260925 10:59:45.921922 20961 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:45.922031 20961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:45.922317 20961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:45.925748 20961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "45371e927edd4971b5382c4b7736d476"
format_stamp: "Formatted at 2026-09-25 10:59:45 on dist-test-slave-nx7s"
server_key: "3f3b0681409a80c948da3030b006570c"
server_key_iv: "417156ffdc63628e40114e448d001d76"
server_key_version: "encryptionkey@0"
I20260925 10:59:45.926524 20961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "45371e927edd4971b5382c4b7736d476"
format_stamp: "Formatted at 2026-09-25 10:59:45 on dist-test-slave-nx7s"
server_key: "3f3b0681409a80c948da3030b006570c"
server_key_iv: "417156ffdc63628e40114e448d001d76"
server_key_version: "encryptionkey@0"
I20260925 10:59:45.931840 20961 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 10:59:45.935449 20976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:45.936908 20961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:59:45.937121 20961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "45371e927edd4971b5382c4b7736d476"
format_stamp: "Formatted at 2026-09-25 10:59:45 on dist-test-slave-nx7s"
server_key: "3f3b0681409a80c948da3030b006570c"
server_key_iv: "417156ffdc63628e40114e448d001d76"
server_key_version: "encryptionkey@0"
I20260925 10:59:45.937319 20961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:45.952630 20961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:45.957010 20961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:45.957275 20961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:45.974573 20961 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36053
I20260925 10:59:45.974602 21027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36053 every 8 connection(s)
I20260925 10:59:45.976023 20961 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:45.979846 21028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:45.986109 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 20961
I20260925 10:59:45.986299 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:45.986991 31994 external_mini_cluster.cc:1428] Setting key 15112cab6ab0aae362f01a1a9a2c7d26
I20260925 10:59:45.991056 21028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476: Bootstrap starting.
I20260925 10:59:45.994786 21028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:45.995992 21028 log.cc:824] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:45.999485 21028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476: No bootstrap required, opened a new log
Sep 25 10:59:45 dist-test-slave-nx7s krb5kdc[20948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 10:59:46.008678 21028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45371e927edd4971b5382c4b7736d476" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36053 } }
I20260925 10:59:46.009140 21028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:46.009239 21028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45371e927edd4971b5382c4b7736d476, State: Initialized, Role: FOLLOWER
I20260925 10:59:46.009975 21028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45371e927edd4971b5382c4b7736d476" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36053 } }
I20260925 10:59:46.010265 21028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:46.010413 21028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:46.010594 21028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:46.011561 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:45.988983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:45817 (local address 127.0.0.1:36053)
0925 10:59:45.989721 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:45.989734 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:45.989775 (+ 41us) server_negotiation.cc:407] Connection header received
0925 10:59:45.991580 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:45.991605 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:45.992037 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:45.992388 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:45.994233 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:45.995676 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:45.997340 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:45.998097 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:45.998247 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:46.003328 (+ 5081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:46.003341 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:46.003360 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:46.003408 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:46.006678 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:46.007806 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:46.007815 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:46.007825 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:46.008012 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:46.008555 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:46.008558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:46.008560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:46.009231 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:46.009575 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:46.010055 (+ 480us) server_negotiation.cc:299] Negotiation successful
0925 10:59:46.010375 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":339,"threads_started":1}
I20260925 10:59:46.013042 21028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45371e927edd4971b5382c4b7736d476" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36053 } }
I20260925 10:59:46.013549 21028 leader_election.cc:302] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45371e927edd4971b5382c4b7736d476; no voters:
I20260925 10:59:46.014168 21028 leader_election.cc:288] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:46.014331 21033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:46.015358 21033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [term 1 LEADER]: Becoming Leader. State: Replica: 45371e927edd4971b5382c4b7736d476, State: Running, Role: LEADER
I20260925 10:59:46.015725 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:46.015909 21033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45371e927edd4971b5382c4b7736d476" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36053 } }
I20260925 10:59:46.016471 21028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:46.018931 21034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45371e927edd4971b5382c4b7736d476" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45371e927edd4971b5382c4b7736d476" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36053 } } }
I20260925 10:59:46.018980 21035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45371e927edd4971b5382c4b7736d476. Latest consensus state: current_term: 1 leader_uuid: "45371e927edd4971b5382c4b7736d476" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45371e927edd4971b5382c4b7736d476" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36053 } } }
I20260925 10:59:46.019421 21035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:46.019431 21034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:46.033301 21041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:46.039367 21041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:46.059645 21041 catalog_manager.cc:1347] Generated new cluster ID: 05966a6896c948d38394130e3c7cacb2
I20260925 10:59:46.059773 21041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:46.066212 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:46.040432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:49757 (local address 127.0.0.1:36053)
0925 10:59:46.040609 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:46.040616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:46.047676 (+ 7060us) server_negotiation.cc:407] Connection header received
0925 10:59:46.047937 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:46.047943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:46.048027 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:46.048103 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:46.050181 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:46.051414 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:46.053429 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:46.054089 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:46.054189 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:46.057674 (+ 3485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:46.057683 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:46.057687 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:46.057735 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:46.061748 (+ 4013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:46.062856 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:46.062862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:46.062865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:46.063019 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:46.063619 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:46.063624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:46.063628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:46.064130 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:46.065412 (+ 1282us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:46.065604 (+ 192us) server_negotiation.cc:299] Negotiation successful
0925 10:59:46.065796 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":36}
I20260925 10:59:46.075913 21041 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:46.077248 21041 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:46.085566 21041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45371e927edd4971b5382c4b7736d476: Generated new TSK 0
I20260925 10:59:46.086473 21041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:46.091985 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 20961
I20260925 10:59:46.093174 21023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:46.183539 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 20961
W20260925 10:59:46.197201 20946 connection.cc:535] client connection to 127.0.0.1:36053 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36053): Connection reset by peer (error 104)
2026-09-25T10:59:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1898 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:46 dist-test-slave-nx7s krb5kdc[21060](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:46 dist-test-slave-nx7s krb5kdc[21060](info): set up 2 sockets
Sep 25 10:59:46 dist-test-slave-nx7s krb5kdc[21060](info): commencing operation
krb5kdc: starting...
W20260925 10:59:47.205459 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:47 dist-test-slave-nx7s krb5kdc[21060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 10:59:47.464179 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:47.784775 21073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:47.785169 21073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:47.785331 21073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:47.792937 21073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:47.793076 21073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:47.793172 21073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:47.793260 21073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:47.793322 21073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:47.803838 21073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41099
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:47.805287 21073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:47.806685 21073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:47.819294 21079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:47.819330 21081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:47.820096 21078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:47.820158 21073 server_base.cc:1034] running on GCE node
I20260925 10:59:47.821058 21073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:47.822564 21073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:47.823834 21073 hybrid_clock.cc:648] HybridClock initialized: now 1790333987823770 us; error 55 us; skew 500 ppm
I20260925 10:59:47.824208 21073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:47 dist-test-slave-nx7s krb5kdc[21060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333987, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:47.833516 21073 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 10:59:47.835294 21073 webserver.cc:460] Webserver started at http://127.0.0.1:42235/ using document root <none> and password file <none>
I20260925 10:59:47.836079 21073 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:47.836205 21073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:47.836508 21073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:47.839699 21073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "72e95acad7a24aef820ac3b0387a6855"
format_stamp: "Formatted at 2026-09-25 10:59:47 on dist-test-slave-nx7s"
server_key: "9d17b87868641bb159cf177ff656c781"
server_key_iv: "54713e61b1e2b281487e2d3e6e6714f7"
server_key_version: "encryptionkey@0"
I20260925 10:59:47.840507 21073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "72e95acad7a24aef820ac3b0387a6855"
format_stamp: "Formatted at 2026-09-25 10:59:47 on dist-test-slave-nx7s"
server_key: "9d17b87868641bb159cf177ff656c781"
server_key_iv: "54713e61b1e2b281487e2d3e6e6714f7"
server_key_version: "encryptionkey@0"
I20260925 10:59:47.846225 21073 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 10:59:47.850090 21088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:47.851548 21073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:59:47.851722 21073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "72e95acad7a24aef820ac3b0387a6855"
format_stamp: "Formatted at 2026-09-25 10:59:47 on dist-test-slave-nx7s"
server_key: "9d17b87868641bb159cf177ff656c781"
server_key_iv: "54713e61b1e2b281487e2d3e6e6714f7"
server_key_version: "encryptionkey@0"
I20260925 10:59:47.851944 21073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:47.867151 21073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:47.871459 21073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:47.871727 21073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:47.889192 21073 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41099
I20260925 10:59:47.889216 21139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41099 every 8 connection(s)
I20260925 10:59:47.890641 21073 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:47.894388 21140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:47.897384 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 21073
I20260925 10:59:47.897527 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:47.897960 31994 external_mini_cluster.cc:1428] Setting key b73d9252424e319b73e53d55dc7cedab
I20260925 10:59:47.906066 21140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855: Bootstrap starting.
I20260925 10:59:47.909274 21140 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:47.910368 21140 log.cc:824] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:47.912144 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:47.899854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:51117 (local address 127.0.0.1:41099)
0925 10:59:47.900687 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:47.900700 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:47.900754 (+ 54us) server_negotiation.cc:407] Connection header received
0925 10:59:47.902633 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:47.902659 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:47.903150 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:47.903589 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:47.904200 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:47.905635 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:47.907150 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:47.907934 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:47.908129 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:47.909509 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:47.909521 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:47.909541 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:47.910149 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:47.910249 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:47.910278 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:47.910437 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:47.910757 (+ 320us) server_negotiation.cc:299] Negotiation successful
0925 10:59:47.910974 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":359,"threads_started":1}
I20260925 10:59:47.913741 21140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855: No bootstrap required, opened a new log
I20260925 10:59:47.916146 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:47.921877 21140 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e95acad7a24aef820ac3b0387a6855" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41099 } }
I20260925 10:59:47.922420 21140 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:47.922528 21140 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72e95acad7a24aef820ac3b0387a6855, State: Initialized, Role: FOLLOWER
I20260925 10:59:47.923182 21140 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e95acad7a24aef820ac3b0387a6855" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41099 } }
I20260925 10:59:47.923400 21140 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:47.923528 21140 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:47.923677 21140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:47.926574 21140 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e95acad7a24aef820ac3b0387a6855" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41099 } }
I20260925 10:59:47.926890 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:47.920519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:37267 (local address 127.0.0.1:41099)
0925 10:59:47.920706 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:47.920712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:47.920942 (+ 230us) server_negotiation.cc:407] Connection header received
0925 10:59:47.921116 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:47.921120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:47.921200 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:47.921309 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:47.921754 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:47.922815 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:47.924284 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:47.925009 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:47.925155 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:47.926157 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:47.926163 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:47.926167 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:47.926189 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:47.926254 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:47.926275 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:47.926387 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:47.926527 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 10:59:47.926590 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 10:59:47.927239 21140 leader_election.cc:302] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72e95acad7a24aef820ac3b0387a6855; no voters:
I20260925 10:59:47.927923 21140 leader_election.cc:288] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:47.928076 21150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:47.928726 21150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [term 1 LEADER]: Becoming Leader. State: Replica: 72e95acad7a24aef820ac3b0387a6855, State: Running, Role: LEADER
I20260925 10:59:47.929376 21150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e95acad7a24aef820ac3b0387a6855" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41099 } }
I20260925 10:59:47.931214 21150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72e95acad7a24aef820ac3b0387a6855. Latest consensus state: current_term: 1 leader_uuid: "72e95acad7a24aef820ac3b0387a6855" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e95acad7a24aef820ac3b0387a6855" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41099 } } }
I20260925 10:59:47.932324 21152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72e95acad7a24aef820ac3b0387a6855" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e95acad7a24aef820ac3b0387a6855" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41099 } } }
I20260925 10:59:47.932662 21150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:47.932744 21152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:47.935781 21140 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:47.936628 21154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:47.942119 21154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:47.951510 21154 catalog_manager.cc:1347] Generated new cluster ID: 3dd24dddf50e4cbca9e1587095d6b6d2
I20260925 10:59:47.951627 21154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:47.960608 21154 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:47.961951 21154 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:47.969615 21154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72e95acad7a24aef820ac3b0387a6855: Generated new TSK 0
I20260925 10:59:47.970446 21154 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:47.982872 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 21073
W20260925 10:59:47.983986 21135 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.136:51117
W20260925 10:59:47.984826 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 21073: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:59:47.984995 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 21073
W20260925 10:59:47.998505 21058 connection.cc:535] client connection to 127.0.0.1:41099 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41099): Connection reset by peer (error 104)
2026-09-25T10:59:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1801 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:48 dist-test-slave-nx7s krb5kdc[21171](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:48 dist-test-slave-nx7s krb5kdc[21171](info): set up 2 sockets
Sep 25 10:59:48 dist-test-slave-nx7s krb5kdc[21171](info): commencing operation
krb5kdc: starting...
W20260925 10:59:49.005744 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:49 dist-test-slave-nx7s krb5kdc[21171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 10:59:49.252220 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37619
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:49.581532 21184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:49.581872 21184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:49.582015 21184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:49.589409 21184 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:49.589536 21184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:49.589610 21184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:49.589694 21184 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:49.589752 21184 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:49.599931 21184 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37619
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:49.601311 21184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:49.602700 21184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:49.614650 21190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:49.614794 21192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:49.615154 21189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:49.615514 21184 server_base.cc:1034] running on GCE node
I20260925 10:59:49.616328 21184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:49.617915 21184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:49.619153 21184 hybrid_clock.cc:648] HybridClock initialized: now 1790333989619096 us; error 43 us; skew 500 ppm
I20260925 10:59:49.619524 21184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:49 dist-test-slave-nx7s krb5kdc[21171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333989, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:49.622663 21184 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 10:59:49.624219 21184 webserver.cc:460] Webserver started at http://127.0.0.1:45079/ using document root <none> and password file <none>
I20260925 10:59:49.625001 21184 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:49.625128 21184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:49.625418 21184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:49.628476 21184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "ba9b57bf21f44c3687acc0844c3b3059"
format_stamp: "Formatted at 2026-09-25 10:59:49 on dist-test-slave-nx7s"
server_key: "695e828cbade86f92a994215e89f555a"
server_key_iv: "a2c37ac3ccab0933d2f14a31cbe323ef"
server_key_version: "encryptionkey@0"
I20260925 10:59:49.629264 21184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "ba9b57bf21f44c3687acc0844c3b3059"
format_stamp: "Formatted at 2026-09-25 10:59:49 on dist-test-slave-nx7s"
server_key: "695e828cbade86f92a994215e89f555a"
server_key_iv: "a2c37ac3ccab0933d2f14a31cbe323ef"
server_key_version: "encryptionkey@0"
I20260925 10:59:49.634574 21184 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:59:49.638161 21199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:49.639678 21184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:59:49.639869 21184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "ba9b57bf21f44c3687acc0844c3b3059"
format_stamp: "Formatted at 2026-09-25 10:59:49 on dist-test-slave-nx7s"
server_key: "695e828cbade86f92a994215e89f555a"
server_key_iv: "a2c37ac3ccab0933d2f14a31cbe323ef"
server_key_version: "encryptionkey@0"
I20260925 10:59:49.640123 21184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:49.646108 21184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:49.650494 21184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:49.650785 21184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:49.668129 21184 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37619
I20260925 10:59:49.668157 21250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37619 every 8 connection(s)
I20260925 10:59:49.669560 21184 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:49.673261 21251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:49.675797 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 21184
I20260925 10:59:49.675971 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:49.676493 31994 external_mini_cluster.cc:1428] Setting key 4374a8a690f4acd300b3683fc2b57f70
I20260925 10:59:49.684707 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:49.678555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:53853 (local address 127.0.0.1:37619)
0925 10:59:49.679383 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:49.679397 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:49.679444 (+ 47us) server_negotiation.cc:407] Connection header received
0925 10:59:49.681180 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:49.681206 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:49.681588 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:49.682019 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:49.683011 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:49.683018 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:49.683033 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:49.683052 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:49.683117 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:49.683139 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:49.683379 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:49.683520 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 10:59:49.683735 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":348,"threads_started":1}
I20260925 10:59:49.685590 21251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059: Bootstrap starting.
I20260925 10:59:49.688364 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:49.689033 21251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:49.690008 21251 log.cc:824] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:49.693030 21251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059: No bootstrap required, opened a new log
I20260925 10:59:49.695322 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:49.692802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:59199 (local address 127.0.0.1:37619)
0925 10:59:49.692979 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:49.692984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:49.693475 (+ 491us) server_negotiation.cc:407] Connection header received
0925 10:59:49.693639 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:49.693645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:49.693700 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:49.693779 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:49.694681 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:49.694687 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:49.694691 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:49.694709 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:49.694753 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:49.694768 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:49.694882 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:49.694989 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 10:59:49.695063 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 10:59:49.701227 21251 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9b57bf21f44c3687acc0844c3b3059" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37619 } }
I20260925 10:59:49.701622 21251 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:49.701714 21251 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba9b57bf21f44c3687acc0844c3b3059, State: Initialized, Role: FOLLOWER
I20260925 10:59:49.702255 21251 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9b57bf21f44c3687acc0844c3b3059" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37619 } }
I20260925 10:59:49.702467 21251 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:49.702597 21251 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:49.702739 21251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:49.704959 21251 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9b57bf21f44c3687acc0844c3b3059" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37619 } }
I20260925 10:59:49.705523 21251 leader_election.cc:302] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba9b57bf21f44c3687acc0844c3b3059; no voters:
I20260925 10:59:49.706110 21251 leader_election.cc:288] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:49.706187 21261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:49.706858 21261 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [term 1 LEADER]: Becoming Leader. State: Replica: ba9b57bf21f44c3687acc0844c3b3059, State: Running, Role: LEADER
I20260925 10:59:49.707367 21261 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9b57bf21f44c3687acc0844c3b3059" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37619 } }
I20260925 10:59:49.708120 21251 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:49.709950 21262 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba9b57bf21f44c3687acc0844c3b3059" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9b57bf21f44c3687acc0844c3b3059" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37619 } } }
I20260925 10:59:49.710378 21262 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:49.710743 21263 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba9b57bf21f44c3687acc0844c3b3059. Latest consensus state: current_term: 1 leader_uuid: "ba9b57bf21f44c3687acc0844c3b3059" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9b57bf21f44c3687acc0844c3b3059" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37619 } } }
I20260925 10:59:49.711030 21263 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:49.712013 21268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:49.716598 21268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:49.726992 21268 catalog_manager.cc:1347] Generated new cluster ID: 518f2f7f5a7a4ab8bbe78d86b80ac461
I20260925 10:59:49.727171 21268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:49.746236 21268 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:49.746752 21268 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:49.756379 21268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba9b57bf21f44c3687acc0844c3b3059: Generated new TSK 0
I20260925 10:59:49.757388 21268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:49.774384 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 21184
W20260925 10:59:49.775794 21246 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.136:53853
W20260925 10:59:49.776489 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 21184: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:59:49.776618 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 21184
2026-09-25T10:59:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1791 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:49 dist-test-slave-nx7s krb5kdc[21282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:49 dist-test-slave-nx7s krb5kdc[21282](info): set up 2 sockets
Sep 25 10:59:49 dist-test-slave-nx7s krb5kdc[21282](info): commencing operation
krb5kdc: starting...
W20260925 10:59:50.789444 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:50 dist-test-slave-nx7s krb5kdc[21282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:59:51.039852 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:35509
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:40313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:35509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:51.373904 21295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:51.374230 21295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:51.374361 21295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:51.381745 21295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:51.381883 21295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:51.381969 21295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:51.382046 21295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:51.382102 21295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:51.392244 21295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:40313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:35509
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:35509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:51.393636 21295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:51.394949 21295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:51.406553 21301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:51.406728 21300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:51.406760 21303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:51.407909 21295 server_base.cc:1034] running on GCE node
I20260925 10:59:51.408767 21295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:51.410328 21295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:51.411547 21295 hybrid_clock.cc:648] HybridClock initialized: now 1790333991411483 us; error 63 us; skew 500 ppm
I20260925 10:59:51.411909 21295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:51 dist-test-slave-nx7s krb5kdc[21282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333991, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:51.415153 21295 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:59:51.416822 21295 webserver.cc:460] Webserver started at http://127.31.62.190:36143/ using document root <none> and password file <none>
I20260925 10:59:51.417572 21295 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:51.417676 21295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:51.417958 21295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:51.421029 21295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "6235ee2230ca441ea4b87934a7badf96"
format_stamp: "Formatted at 2026-09-25 10:59:51 on dist-test-slave-nx7s"
server_key: "34130fe81506c0957278f25eac886c74"
server_key_iv: "bccdd33566f34c3fd4455549e4a61c9e"
server_key_version: "encryptionkey@0"
I20260925 10:59:51.421777 21295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "6235ee2230ca441ea4b87934a7badf96"
format_stamp: "Formatted at 2026-09-25 10:59:51 on dist-test-slave-nx7s"
server_key: "34130fe81506c0957278f25eac886c74"
server_key_iv: "bccdd33566f34c3fd4455549e4a61c9e"
server_key_version: "encryptionkey@0"
I20260925 10:59:51.427299 21295 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:59:51.431139 21310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:51.432659 21295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 10:59:51.432898 21295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "6235ee2230ca441ea4b87934a7badf96"
format_stamp: "Formatted at 2026-09-25 10:59:51 on dist-test-slave-nx7s"
server_key: "34130fe81506c0957278f25eac886c74"
server_key_iv: "bccdd33566f34c3fd4455549e4a61c9e"
server_key_version: "encryptionkey@0"
I20260925 10:59:51.433111 21295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:51.445904 21295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:51.450359 21295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:51.450625 21295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:51.468247 21295 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:35509
I20260925 10:59:51.468255 21361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:35509 every 8 connection(s)
I20260925 10:59:51.469728 21295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:51.473639 21362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:51.473650 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 21295
I20260925 10:59:51.473794 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:51.474225 31994 external_mini_cluster.cc:1428] Setting key 1e3925c23f2ceabf5852d87486a2465e
I20260925 10:59:51.487880 21362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96: Bootstrap starting.
Sep 25 10:59:51 dist-test-slave-nx7s krb5kdc[21282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:59:51.491173 21362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:51.492237 21362 log.cc:824] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:51.495010 21362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96: No bootstrap required, opened a new log
I20260925 10:59:51.497071 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:51.475965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52163 (local address 127.31.62.190:35509)
0925 10:59:51.476824 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:51.476840 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:51.476914 (+ 74us) server_negotiation.cc:407] Connection header received
0925 10:59:51.478474 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:51.478499 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:51.478952 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:51.479313 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:51.481127 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:51.482395 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:51.484136 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:51.484933 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:51.485082 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:51.489726 (+ 4644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:51.489735 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:51.489751 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:51.489802 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:51.492697 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:51.493533 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:51.493541 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:51.493548 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:51.493713 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:51.494202 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:51.494206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:51.494208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:51.494737 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:51.495052 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:51.495586 (+ 534us) server_negotiation.cc:299] Negotiation successful
0925 10:59:51.495916 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":409,"threads_started":1}
I20260925 10:59:51.501459 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:51.503698 21362 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6235ee2230ca441ea4b87934a7badf96" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35509 } }
I20260925 10:59:51.504134 21362 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:51.504242 21362 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6235ee2230ca441ea4b87934a7badf96, State: Initialized, Role: FOLLOWER
I20260925 10:59:51.504967 21362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6235ee2230ca441ea4b87934a7badf96" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35509 } }
I20260925 10:59:51.505192 21362 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:51.505324 21362 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:51.505479 21362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:51.507973 21362 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6235ee2230ca441ea4b87934a7badf96" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35509 } }
I20260925 10:59:51.508584 21362 leader_election.cc:302] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6235ee2230ca441ea4b87934a7badf96; no voters:
I20260925 10:59:51.509232 21362 leader_election.cc:288] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:51.509397 21372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:51.511168 21362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:51.511333 21372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [term 1 LEADER]: Becoming Leader. State: Replica: 6235ee2230ca441ea4b87934a7badf96, State: Running, Role: LEADER
I20260925 10:59:51.511926 21372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6235ee2230ca441ea4b87934a7badf96" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35509 } }
I20260925 10:59:51.514792 21374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6235ee2230ca441ea4b87934a7badf96. Latest consensus state: current_term: 1 leader_uuid: "6235ee2230ca441ea4b87934a7badf96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6235ee2230ca441ea4b87934a7badf96" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35509 } } }
I20260925 10:59:51.515208 21374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:51.515751 21373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6235ee2230ca441ea4b87934a7badf96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6235ee2230ca441ea4b87934a7badf96" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35509 } } }
I20260925 10:59:51.516039 21373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:51.519980 21379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:51.522130 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:51.505769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56607 (local address 127.31.62.190:35509)
0925 10:59:51.505951 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:51.505957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:51.506380 (+ 423us) server_negotiation.cc:407] Connection header received
0925 10:59:51.506534 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:51.506540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:51.506622 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:51.506739 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:51.508097 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:51.509239 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:51.511033 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:51.511698 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:51.511834 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:51.515030 (+ 3196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:51.515037 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:51.515042 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:51.515091 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:51.518891 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:51.519889 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:51.519898 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:51.519902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:51.520042 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:51.520530 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:51.520533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:51.520535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:51.520893 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:51.521207 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:51.521659 (+ 452us) server_negotiation.cc:299] Negotiation successful
0925 10:59:51.521803 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 10:59:51.525812 21379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:51.534577 21379 catalog_manager.cc:1347] Generated new cluster ID: 51dac48a3f434f55ac8b6e3742699d49
I20260925 10:59:51.534729 21379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:51.552176 21379 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:51.553555 21379 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:51.561324 21379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6235ee2230ca441ea4b87934a7badf96: Generated new TSK 0
I20260925 10:59:51.562465 21379 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:51.572712 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 21295
I20260925 10:59:51.573941 21357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:51.664258 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 21295
W20260925 10:59:51.677763 21280 connection.cc:535] client connection to 127.31.62.190:35509 recv error: Network error: failed to read from TLS socket (remote: 127.31.62.190:35509): Connection reset by peer (error 104)
2026-09-25T10:59:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1887 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:51 dist-test-slave-nx7s krb5kdc[21394](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:51 dist-test-slave-nx7s krb5kdc[21394](info): set up 2 sockets
Sep 25 10:59:51 dist-test-slave-nx7s krb5kdc[21394](info): commencing operation
krb5kdc: starting...
W20260925 10:59:52.681605 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.005s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:52 dist-test-slave-nx7s krb5kdc[21394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:59:52.936349 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:35709
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:38017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:35709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:53.262252 21407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:53.262588 21407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:53.262683 21407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:53.270161 21407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:53.270290 21407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:53.270336 21407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:53.270401 21407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:53.270444 21407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:53.280787 21407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:38017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:35709
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:35709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:53.282094 21407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:53.283344 21407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:53.295233 21412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:53.295166 21413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:53.295209 21415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:53.296052 21407 server_base.cc:1034] running on GCE node
I20260925 10:59:53.297592 21407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:53.299291 21407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:53.300531 21407 hybrid_clock.cc:648] HybridClock initialized: now 1790333993300474 us; error 42 us; skew 500 ppm
I20260925 10:59:53.301033 21407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:53 dist-test-slave-nx7s krb5kdc[21394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333993, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:53.308645 21407 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:59:53.310438 21407 webserver.cc:460] Webserver started at http://127.31.62.190:40587/ using document root <none> and password file <none>
I20260925 10:59:53.311168 21407 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:53.311282 21407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:53.311571 21407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:53.314713 21407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "12d0993acbff43ed8a612dc3b58ab006"
format_stamp: "Formatted at 2026-09-25 10:59:53 on dist-test-slave-nx7s"
server_key: "7485d2a0ec4915786dd5523657549921"
server_key_iv: "9f55e5224227974a8ce750db958851db"
server_key_version: "encryptionkey@0"
I20260925 10:59:53.315536 21407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "12d0993acbff43ed8a612dc3b58ab006"
format_stamp: "Formatted at 2026-09-25 10:59:53 on dist-test-slave-nx7s"
server_key: "7485d2a0ec4915786dd5523657549921"
server_key_iv: "9f55e5224227974a8ce750db958851db"
server_key_version: "encryptionkey@0"
I20260925 10:59:53.321062 21407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260925 10:59:53.324792 21422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:53.326386 21407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 10:59:53.326571 21407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "12d0993acbff43ed8a612dc3b58ab006"
format_stamp: "Formatted at 2026-09-25 10:59:53 on dist-test-slave-nx7s"
server_key: "7485d2a0ec4915786dd5523657549921"
server_key_iv: "9f55e5224227974a8ce750db958851db"
server_key_version: "encryptionkey@0"
I20260925 10:59:53.326795 21407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:53.354866 21407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:53.360807 21407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:53.361155 21407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:53.377986 21407 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:35709
I20260925 10:59:53.378005 21473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:35709 every 8 connection(s)
I20260925 10:59:53.379343 21407 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:53.380805 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 21407
I20260925 10:59:53.381000 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:53.381541 31994 external_mini_cluster.cc:1428] Setting key 5eaff88ac6633f5247ff781c7d7eb30b
I20260925 10:59:53.383993 21474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:53.397142 21477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:53.383377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58917 (local address 127.31.62.190:35709)
0925 10:59:53.384144 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:53.384159 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:53.384208 (+ 49us) server_negotiation.cc:407] Connection header received
0925 10:59:53.386493 (+ 2285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:53.386522 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:53.387059 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:53.387532 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:53.388043 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:53.390079 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:53.391537 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:53.392554 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:53.392728 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:53.394184 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:53.394196 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:53.394215 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:53.394979 (+ 764us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:53.395079 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:53.395118 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:53.395293 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:53.395576 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 10:59:53.395779 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":326,"threads_started":1}
I20260925 10:59:53.397950 21474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006: Bootstrap starting.
I20260925 10:59:53.401474 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:53.401600 21474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:53.402822 21474 log.cc:824] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:53.406369 21474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006: No bootstrap required, opened a new log
I20260925 10:59:53.413523 21477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:53.406555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56029 (local address 127.31.62.190:35709)
0925 10:59:53.406735 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:53.406741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:53.407128 (+ 387us) server_negotiation.cc:407] Connection header received
0925 10:59:53.407301 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:53.407307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:53.407384 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:53.407504 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:53.408070 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:53.409303 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:53.410650 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:53.411343 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:53.411500 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:53.412702 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:53.412709 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:53.412714 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:53.412737 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:53.412791 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:53.412812 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:53.412957 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:53.413107 (+ 150us) server_negotiation.cc:299] Negotiation successful
0925 10:59:53.413164 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 10:59:53.416072 21474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0993acbff43ed8a612dc3b58ab006" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35709 } }
I20260925 10:59:53.416471 21474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:53.416550 21474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12d0993acbff43ed8a612dc3b58ab006, State: Initialized, Role: FOLLOWER
I20260925 10:59:53.417128 21474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0993acbff43ed8a612dc3b58ab006" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35709 } }
I20260925 10:59:53.417335 21474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:53.417438 21474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:53.417570 21474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:53.419718 21474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0993acbff43ed8a612dc3b58ab006" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35709 } }
I20260925 10:59:53.420204 21474 leader_election.cc:302] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12d0993acbff43ed8a612dc3b58ab006; no voters:
I20260925 10:59:53.420810 21474 leader_election.cc:288] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:53.420998 21484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:53.422339 21484 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [term 1 LEADER]: Becoming Leader. State: Replica: 12d0993acbff43ed8a612dc3b58ab006, State: Running, Role: LEADER
I20260925 10:59:53.422851 21484 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0993acbff43ed8a612dc3b58ab006" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35709 } }
I20260925 10:59:53.423255 21474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:53.425450 21485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12d0993acbff43ed8a612dc3b58ab006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0993acbff43ed8a612dc3b58ab006" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35709 } } }
I20260925 10:59:53.425773 21485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:53.427028 21486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12d0993acbff43ed8a612dc3b58ab006. Latest consensus state: current_term: 1 leader_uuid: "12d0993acbff43ed8a612dc3b58ab006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d0993acbff43ed8a612dc3b58ab006" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35709 } } }
I20260925 10:59:53.427385 21486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:53.429234 21492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:53.434530 21492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:53.443638 21492 catalog_manager.cc:1347] Generated new cluster ID: 73ad080feb4048d0ae1b07e4c1e2c924
I20260925 10:59:53.443758 21492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:53.455622 21492 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:53.457635 21492 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:53.470062 21492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12d0993acbff43ed8a612dc3b58ab006: Generated new TSK 0
I20260925 10:59:53.471026 21492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:53.485805 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 21407
W20260925 10:59:53.486958 21469 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58917
W20260925 10:59:53.487758 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 21407: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:59:53.487898 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 21407
W20260925 10:59:53.501132 21392 connection.cc:535] client connection to 127.31.62.190:35709 recv error: Network error: failed to read from TLS socket (remote: 127.31.62.190:35709): Connection reset by peer (error 104)
2026-09-25T10:59:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1823 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:53 dist-test-slave-nx7s krb5kdc[21505](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:53 dist-test-slave-nx7s krb5kdc[21505](info): set up 2 sockets
Sep 25 10:59:53 dist-test-slave-nx7s krb5kdc[21505](info): commencing operation
krb5kdc: starting...
W20260925 10:59:54.513499 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:54 dist-test-slave-nx7s krb5kdc[21505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:59:54.768348 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:36487
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:38813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:36487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:55.097064 21518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:55.097368 21518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:55.097497 21518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:55.104923 21518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:55.105043 21518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:55.105114 21518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:55.105190 21518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:55.105252 21518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:55.115422 21518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:38813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:36487
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:36487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:55.116755 21518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:55.118089 21518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:55.129840 21526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:55.129913 21524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:55.130306 21523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:55.130796 21518 server_base.cc:1034] running on GCE node
I20260925 10:59:55.131559 21518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:55.133087 21518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:55.134294 21518 hybrid_clock.cc:648] HybridClock initialized: now 1790333995134238 us; error 45 us; skew 500 ppm
I20260925 10:59:55.134630 21518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:55 dist-test-slave-nx7s krb5kdc[21505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333995, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:55.137941 21518 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:59:55.139534 21518 webserver.cc:460] Webserver started at http://127.31.62.190:43835/ using document root <none> and password file <none>
I20260925 10:59:55.140199 21518 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:55.140300 21518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:55.140561 21518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:55.143566 21518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "266b0880a2a14759b8cb97d3e6160707"
format_stamp: "Formatted at 2026-09-25 10:59:55 on dist-test-slave-nx7s"
server_key: "b6c08853447ecaaea4dfae107c94187a"
server_key_iv: "64443b17f6929b83127909a0e90fe3ff"
server_key_version: "encryptionkey@0"
I20260925 10:59:55.144253 21518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "266b0880a2a14759b8cb97d3e6160707"
format_stamp: "Formatted at 2026-09-25 10:59:55 on dist-test-slave-nx7s"
server_key: "b6c08853447ecaaea4dfae107c94187a"
server_key_iv: "64443b17f6929b83127909a0e90fe3ff"
server_key_version: "encryptionkey@0"
I20260925 10:59:55.149520 21518 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260925 10:59:55.153200 21533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:55.154798 21518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 10:59:55.155046 21518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "266b0880a2a14759b8cb97d3e6160707"
format_stamp: "Formatted at 2026-09-25 10:59:55 on dist-test-slave-nx7s"
server_key: "b6c08853447ecaaea4dfae107c94187a"
server_key_iv: "64443b17f6929b83127909a0e90fe3ff"
server_key_version: "encryptionkey@0"
I20260925 10:59:55.155262 21518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:55.161459 21518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:55.165943 21518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:55.166208 21518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:55.183889 21518 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:36487
I20260925 10:59:55.183987 21584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:36487 every 8 connection(s)
I20260925 10:59:55.185514 21518 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:55.189790 21585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:55.193651 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 21518
I20260925 10:59:55.193826 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:55.194402 31994 external_mini_cluster.cc:1428] Setting key 9ceaa2796e54e0848ef5843a56be3250
I20260925 10:59:55.202540 21588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:55.196409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32917 (local address 127.31.62.190:36487)
0925 10:59:55.197223 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:55.197235 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:55.197280 (+ 45us) server_negotiation.cc:407] Connection header received
0925 10:59:55.198955 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:55.198980 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:55.199383 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:55.199789 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:55.200806 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:55.200813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:55.200827 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:55.200842 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:55.200944 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:55.200970 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:55.201181 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:55.201323 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 10:59:55.201542 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":338,"threads_started":1}
I20260925 10:59:55.202682 21585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707: Bootstrap starting.
I20260925 10:59:55.206534 21585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:55.206730 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:55.207633 21585 log.cc:824] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:55.210906 21585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707: No bootstrap required, opened a new log
I20260925 10:59:55.213792 21588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:55.211293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59975 (local address 127.31.62.190:36487)
0925 10:59:55.211500 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:55.211507 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:55.211861 (+ 354us) server_negotiation.cc:407] Connection header received
0925 10:59:55.212043 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:55.212049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:55.212105 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:55.212193 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:55.213135 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:55.213140 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:55.213145 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:55.213163 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:55.213209 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:55.213224 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:55.213331 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:55.213432 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 10:59:55.213504 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 10:59:55.219028 21585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "266b0880a2a14759b8cb97d3e6160707" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36487 } }
I20260925 10:59:55.219440 21585 raft_consensus.cc:374] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:55.219538 21585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 266b0880a2a14759b8cb97d3e6160707, State: Initialized, Role: FOLLOWER
I20260925 10:59:55.220130 21585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "266b0880a2a14759b8cb97d3e6160707" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36487 } }
I20260925 10:59:55.220350 21585 raft_consensus.cc:388] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:55.220480 21585 raft_consensus.cc:482] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:55.220618 21585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:55.222869 21585 raft_consensus.cc:504] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "266b0880a2a14759b8cb97d3e6160707" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36487 } }
I20260925 10:59:55.223413 21585 leader_election.cc:302] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 266b0880a2a14759b8cb97d3e6160707; no voters:
I20260925 10:59:55.224040 21585 leader_election.cc:288] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:55.224156 21595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:55.224812 21595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [term 1 LEADER]: Becoming Leader. State: Replica: 266b0880a2a14759b8cb97d3e6160707, State: Running, Role: LEADER
I20260925 10:59:55.225473 21595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "266b0880a2a14759b8cb97d3e6160707" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36487 } }
I20260925 10:59:55.226433 21585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:55.228566 21597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 266b0880a2a14759b8cb97d3e6160707. Latest consensus state: current_term: 1 leader_uuid: "266b0880a2a14759b8cb97d3e6160707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "266b0880a2a14759b8cb97d3e6160707" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36487 } } }
I20260925 10:59:55.229012 21597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:55.228663 21596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "266b0880a2a14759b8cb97d3e6160707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "266b0880a2a14759b8cb97d3e6160707" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 36487 } } }
I20260925 10:59:55.229579 21596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707 [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:55.231827 21601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:55.237210 21601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:55.247653 21601 catalog_manager.cc:1347] Generated new cluster ID: 220517b6f15d440b92041b04a2ac3efb
I20260925 10:59:55.247788 21601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:55.278194 21601 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:55.278687 21601 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:55.288718 21601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 266b0880a2a14759b8cb97d3e6160707: Generated new TSK 0
I20260925 10:59:55.289710 21601 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:55.303666 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 21518
W20260925 10:59:55.304791 21580 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:32917
W20260925 10:59:55.305546 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 21518: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:59:55.305680 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 21518
2026-09-25T10:59:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1818 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:55 dist-test-slave-nx7s krb5kdc[21616](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:55 dist-test-slave-nx7s krb5kdc[21616](info): set up 2 sockets
Sep 25 10:59:55 dist-test-slave-nx7s krb5kdc[21616](info): commencing operation
krb5kdc: starting...
W20260925 10:59:56.321410 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:56 dist-test-slave-nx7s krb5kdc[21616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:59:56.575698 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:38495
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:35797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:38495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:56.901360 21629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:56.901665 21629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:56.901809 21629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:56.909171 21629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:56.909293 21629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:56.909340 21629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:56.909422 21629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:56.909487 21629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:56.919690 21629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:35797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:38495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:38495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:56.921118 21629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:56.922438 21629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:56.934231 21634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:56.934312 21635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:56.934264 21637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:56.935616 21629 server_base.cc:1034] running on GCE node
I20260925 10:59:56.936447 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:56.938083 21629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:56.939306 21629 hybrid_clock.cc:648] HybridClock initialized: now 1790333996939243 us; error 52 us; skew 500 ppm
I20260925 10:59:56.939678 21629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:56 dist-test-slave-nx7s krb5kdc[21616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333996, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:56.942713 21629 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:59:56.944254 21629 webserver.cc:460] Webserver started at http://127.31.62.190:40031/ using document root <none> and password file <none>
I20260925 10:59:56.944994 21629 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:56.945116 21629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:56.945408 21629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:56.948369 21629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "09439188bd014021880b252615c9663e"
format_stamp: "Formatted at 2026-09-25 10:59:56 on dist-test-slave-nx7s"
server_key: "1486701eb90e2b1d8b5502f1425f9c91"
server_key_iv: "a81083885ec5c5c7d81ed235af37da54"
server_key_version: "encryptionkey@0"
I20260925 10:59:56.949173 21629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "09439188bd014021880b252615c9663e"
format_stamp: "Formatted at 2026-09-25 10:59:56 on dist-test-slave-nx7s"
server_key: "1486701eb90e2b1d8b5502f1425f9c91"
server_key_iv: "a81083885ec5c5c7d81ed235af37da54"
server_key_version: "encryptionkey@0"
I20260925 10:59:56.954389 21629 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 10:59:56.957830 21644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:56.959280 21629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260925 10:59:56.959462 21629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "09439188bd014021880b252615c9663e"
format_stamp: "Formatted at 2026-09-25 10:59:56 on dist-test-slave-nx7s"
server_key: "1486701eb90e2b1d8b5502f1425f9c91"
server_key_iv: "a81083885ec5c5c7d81ed235af37da54"
server_key_version: "encryptionkey@0"
I20260925 10:59:56.959646 21629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:56.977491 21629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:56.981493 21629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:56.981760 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:56.999454 21629 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:38495
I20260925 10:59:56.999476 21695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:38495 every 8 connection(s)
I20260925 10:59:57.000806 21629 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:57.004679 21696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:57.009461 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 21629
I20260925 10:59:57.009645 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:57.010193 31994 external_mini_cluster.cc:1428] Setting key 3eac5a3493240137a17f28db6875b6bb
I20260925 10:59:57.016363 21696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e: Bootstrap starting.
I20260925 10:59:57.020040 21696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:57.021248 21696 log.cc:824] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:57.024706 21696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e: No bootstrap required, opened a new log
Sep 25 10:59:57 dist-test-slave-nx7s krb5kdc[21616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 10:59:57.032522 21696 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09439188bd014021880b252615c9663e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38495 } }
I20260925 10:59:57.032922 21696 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:57.033039 21696 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09439188bd014021880b252615c9663e, State: Initialized, Role: FOLLOWER
I20260925 10:59:57.033396 21699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:57.012046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50915 (local address 127.31.62.190:38495)
0925 10:59:57.012874 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:57.012891 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:57.012941 (+ 50us) server_negotiation.cc:407] Connection header received
0925 10:59:57.014795 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:57.014827 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:57.015245 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:57.015571 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:57.017330 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:57.018745 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:57.020457 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:57.021218 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:57.021380 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:57.025968 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:57.025977 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:57.025993 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:57.026034 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:57.029209 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:57.029896 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:57.029903 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:57.029913 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:57.030082 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:57.030456 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:57.030460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:57.030462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:57.031051 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:57.031347 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:57.031805 (+ 458us) server_negotiation.cc:299] Negotiation successful
0925 10:59:57.032242 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":359,"threads_started":1}
I20260925 10:59:57.033655 21696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09439188bd014021880b252615c9663e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38495 } }
I20260925 10:59:57.033890 21696 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:57.034017 21696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:57.034179 21696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:57.036563 21696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09439188bd014021880b252615c9663e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38495 } }
I20260925 10:59:57.037047 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:57.037097 21696 leader_election.cc:302] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09439188bd014021880b252615c9663e; no voters:
I20260925 10:59:57.037639 21696 leader_election.cc:288] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:57.037773 21701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:57.039454 21696 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:57.041134 21701 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [term 1 LEADER]: Becoming Leader. State: Replica: 09439188bd014021880b252615c9663e, State: Running, Role: LEADER
I20260925 10:59:57.041826 21701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09439188bd014021880b252615c9663e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38495 } }
I20260925 10:59:57.045068 21703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09439188bd014021880b252615c9663e. Latest consensus state: current_term: 1 leader_uuid: "09439188bd014021880b252615c9663e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09439188bd014021880b252615c9663e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38495 } } }
I20260925 10:59:57.045058 21702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09439188bd014021880b252615c9663e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09439188bd014021880b252615c9663e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 38495 } } }
I20260925 10:59:57.045570 21702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:57.045567 21703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:57.049401 21711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:57.056708 21711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:57.071914 21699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:57.049028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39497 (local address 127.31.62.190:38495)
0925 10:59:57.049220 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:57.049226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:57.053516 (+ 4290us) server_negotiation.cc:407] Connection header received
0925 10:59:57.053803 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:57.053809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:57.053890 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:57.053976 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:57.056108 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:57.057357 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:57.059706 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:57.060374 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:57.060485 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:57.064112 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:57.064120 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:57.064126 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 10:59:57.064172 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:57.068283 (+ 4111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:57.069295 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:57.069302 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:57.069317 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:57.069482 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 10:59:57.069947 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 10:59:57.069956 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 10:59:57.069959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 10:59:57.070396 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:57.070626 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:57.071239 (+ 613us) server_negotiation.cc:299] Negotiation successful
0925 10:59:57.071503 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 10:59:57.078104 21711 catalog_manager.cc:1347] Generated new cluster ID: 6df7ad10c495448f9fe1bbc2f5d0a547
I20260925 10:59:57.078224 21711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:57.096530 21711 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:57.097874 21711 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:57.104635 21711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09439188bd014021880b252615c9663e: Generated new TSK 0
I20260925 10:59:57.105854 21711 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:57.116739 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 21629
I20260925 10:59:57.117815 21691 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 10:59:57.210448 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 21629
2026-09-25T10:59:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1903 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:57 dist-test-slave-nx7s krb5kdc[21728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:57 dist-test-slave-nx7s krb5kdc[21728](info): set up 2 sockets
Sep 25 10:59:57 dist-test-slave-nx7s krb5kdc[21728](info): commencing operation
krb5kdc: starting...
W20260925 10:59:58.233500 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 10:59:58 dist-test-slave-nx7s krb5kdc[21728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T10:59:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T10:59:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 10:59:58.492362 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41571
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:45037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:41571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 10:59:58.817268 21741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 10:59:58.817588 21741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 10:59:58.817725 21741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 10:59:58.825060 21741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 10:59:58.825177 21741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 10:59:58.825250 21741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 10:59:58.825321 21741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 10:59:58.825378 21741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 10:59:58.835361 21741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:45037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:41571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 10:59:58.836661 21741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 10:59:58.838050 21741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 10:59:58.849743 21747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:58.849805 21749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 10:59:58.849767 21746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 10:59:58.850653 21741 server_base.cc:1034] running on GCE node
I20260925 10:59:58.851459 21741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 10:59:58.852937 21741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 10:59:58.854148 21741 hybrid_clock.cc:648] HybridClock initialized: now 1790333998854093 us; error 44 us; skew 500 ppm
I20260925 10:59:58.854492 21741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 10:59:58 dist-test-slave-nx7s krb5kdc[21728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790333998, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 10:59:58.862999 21741 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 10:59:58.864629 21741 webserver.cc:460] Webserver started at http://127.31.62.190:39633/ using document root <none> and password file <none>
I20260925 10:59:58.865439 21741 fs_manager.cc:359] Metadata directory not provided
I20260925 10:59:58.865551 21741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 10:59:58.865828 21741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 10:59:58.869088 21741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "94953767fa8d4394aa61ae7185fe267c"
format_stamp: "Formatted at 2026-09-25 10:59:58 on dist-test-slave-nx7s"
server_key: "e3a7522475bc3514ead9d9a7a502f7f2"
server_key_iv: "386b2dc85bd453df7ee244cbcdbe7965"
server_key_version: "encryptionkey@0"
I20260925 10:59:58.870096 21741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "94953767fa8d4394aa61ae7185fe267c"
format_stamp: "Formatted at 2026-09-25 10:59:58 on dist-test-slave-nx7s"
server_key: "e3a7522475bc3514ead9d9a7a502f7f2"
server_key_iv: "386b2dc85bd453df7ee244cbcdbe7965"
server_key_version: "encryptionkey@0"
I20260925 10:59:58.875643 21741 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 10:59:58.879168 21756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 10:59:58.880584 21741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 10:59:58.880724 21741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "94953767fa8d4394aa61ae7185fe267c"
format_stamp: "Formatted at 2026-09-25 10:59:58 on dist-test-slave-nx7s"
server_key: "e3a7522475bc3514ead9d9a7a502f7f2"
server_key_iv: "386b2dc85bd453df7ee244cbcdbe7965"
server_key_version: "encryptionkey@0"
I20260925 10:59:58.880986 21741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 10:59:58.898972 21741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 10:59:58.903133 21741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 10:59:58.903360 21741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 10:59:58.920275 21741 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:41571
I20260925 10:59:58.920346 21807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:41571 every 8 connection(s)
I20260925 10:59:58.921651 21741 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 10:59:58.925487 21808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 10:59:58.926618 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 21741
I20260925 10:59:58.926759 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 10:59:58.927160 31994 external_mini_cluster.cc:1428] Setting key c98d780e5f961f3ec0f3f38d8f28ddd8
I20260925 10:59:58.939680 21808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c: Bootstrap starting.
I20260925 10:59:58.942116 21811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:58.929130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47225 (local address 127.31.62.190:41571)
0925 10:59:58.930035 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:58.930051 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:58.930101 (+ 50us) server_negotiation.cc:407] Connection header received
0925 10:59:58.931940 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:58.931973 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:58.932527 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:58.933014 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:58.933508 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:58.935312 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:58.936750 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:58.937881 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:58.938046 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:58.939276 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:58.939287 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:58.939305 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:58.940011 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:58.940108 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:58.940141 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:58.940255 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:58.940478 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 10:59:58.940747 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":400,"threads_started":1}
I20260925 10:59:58.943370 21808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c: Neither blocks nor log segments found. Creating new log.
I20260925 10:59:58.944669 21808 log.cc:824] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c: Log is configured to *not* fsync() on all Append() calls
I20260925 10:59:58.946275 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 10:59:58.947991 21808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c: No bootstrap required, opened a new log
I20260925 10:59:58.957476 21808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94953767fa8d4394aa61ae7185fe267c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41571 } }
I20260925 10:59:58.957962 21811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 10:59:58.951253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33515 (local address 127.31.62.190:41571)
0925 10:59:58.951473 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0925 10:59:58.951484 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 10:59:58.952126 (+ 642us) server_negotiation.cc:407] Connection header received
0925 10:59:58.952273 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 10:59:58.952278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 10:59:58.952355 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 10:59:58.952458 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 10:59:58.952973 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:58.954079 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:58.955420 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 10:59:58.956113 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 10:59:58.956249 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 10:59:58.957256 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 10:59:58.957262 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 10:59:58.957267 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 10:59:58.957288 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 10:59:58.957335 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 10:59:58.957356 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 10:59:58.957445 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 10:59:58.957559 (+ 114us) server_negotiation.cc:299] Negotiation successful
0925 10:59:58.957647 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 10:59:58.958007 21808 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 10:59:58.958109 21808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94953767fa8d4394aa61ae7185fe267c, State: Initialized, Role: FOLLOWER
I20260925 10:59:58.958782 21808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94953767fa8d4394aa61ae7185fe267c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41571 } }
I20260925 10:59:58.959008 21808 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 10:59:58.959136 21808 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 10:59:58.959314 21808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [term 0 FOLLOWER]: Advancing to term 1
I20260925 10:59:58.961565 21808 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94953767fa8d4394aa61ae7185fe267c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41571 } }
I20260925 10:59:58.962095 21808 leader_election.cc:302] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94953767fa8d4394aa61ae7185fe267c; no voters:
I20260925 10:59:58.962756 21808 leader_election.cc:288] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 10:59:58.962867 21818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 10:59:58.963495 21818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [term 1 LEADER]: Becoming Leader. State: Replica: 94953767fa8d4394aa61ae7185fe267c, State: Running, Role: LEADER
I20260925 10:59:58.964066 21818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94953767fa8d4394aa61ae7185fe267c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41571 } }
I20260925 10:59:58.964753 21808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 10:59:58.966578 21819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94953767fa8d4394aa61ae7185fe267c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94953767fa8d4394aa61ae7185fe267c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41571 } } }
I20260925 10:59:58.967231 21819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:58.967021 21820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94953767fa8d4394aa61ae7185fe267c. Latest consensus state: current_term: 1 leader_uuid: "94953767fa8d4394aa61ae7185fe267c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94953767fa8d4394aa61ae7185fe267c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41571 } } }
I20260925 10:59:58.967334 21820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c [sys.catalog]: This master's current role is: LEADER
I20260925 10:59:58.968631 21826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 10:59:58.973843 21826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 10:59:58.982720 21826 catalog_manager.cc:1347] Generated new cluster ID: 332245f7aa9647869de780c9043cb4bd
I20260925 10:59:58.982842 21826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 10:59:59.013718 21826 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 10:59:59.015286 21826 catalog_manager.cc:1504] Loading token signing keys...
I20260925 10:59:59.022051 21826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94953767fa8d4394aa61ae7185fe267c: Generated new TSK 0
I20260925 10:59:59.022900 21826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 10:59:59.029755 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 21741
W20260925 10:59:59.030864 21803 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47225
W20260925 10:59:59.031592 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 21741: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 10:59:59.031714 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 21741
2026-09-25T10:59:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1821 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 10:59:59 dist-test-slave-nx7s krb5kdc[21839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 10:59:59 dist-test-slave-nx7s krb5kdc[21839](info): set up 2 sockets
Sep 25 10:59:59 dist-test-slave-nx7s krb5kdc[21839](info): commencing operation
krb5kdc: starting...
W20260925 11:00:00.041486 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 11:00:00 dist-test-slave-nx7s krb5kdc[21839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:00:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:00:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 11:00:00.289868 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41159
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:41159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:00.616154 21852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:00.616473 21852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:00.616612 21852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:00.624179 21852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:00.624320 21852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:00.624402 21852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:00.624495 21852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:00:00.624557 21852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:00.634624 21852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:41159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:41159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:00.635917 21852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:00.637246 21852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:00.648715 21857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:00.648749 21858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:00.648790 21860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:00.649312 21852 server_base.cc:1034] running on GCE node
I20260925 11:00:00.650211 21852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:00.651675 21852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:00.652905 21852 hybrid_clock.cc:648] HybridClock initialized: now 1790334000652876 us; error 91 us; skew 500 ppm
I20260925 11:00:00.653262 21852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:00 dist-test-slave-nx7s krb5kdc[21839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334000, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:00.656335 21852 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 11:00:00.657886 21852 webserver.cc:460] Webserver started at http://127.31.62.190:35855/ using document root <none> and password file <none>
I20260925 11:00:00.658561 21852 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:00.658677 21852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:00.658968 21852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:00.661963 21852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "d61c9f72d16245ad97d041740a4ab44c"
format_stamp: "Formatted at 2026-09-25 11:00:00 on dist-test-slave-nx7s"
server_key: "19e78889fa18dc0e3df7ad68e2b2709f"
server_key_iv: "6b8c184759715865ceca1ddfed0a5374"
server_key_version: "encryptionkey@0"
I20260925 11:00:00.662715 21852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "d61c9f72d16245ad97d041740a4ab44c"
format_stamp: "Formatted at 2026-09-25 11:00:00 on dist-test-slave-nx7s"
server_key: "19e78889fa18dc0e3df7ad68e2b2709f"
server_key_iv: "6b8c184759715865ceca1ddfed0a5374"
server_key_version: "encryptionkey@0"
I20260925 11:00:00.667789 21852 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 11:00:00.671168 21867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:00.672657 21852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:00:00.672827 21852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "d61c9f72d16245ad97d041740a4ab44c"
format_stamp: "Formatted at 2026-09-25 11:00:00 on dist-test-slave-nx7s"
server_key: "19e78889fa18dc0e3df7ad68e2b2709f"
server_key_iv: "6b8c184759715865ceca1ddfed0a5374"
server_key_version: "encryptionkey@0"
I20260925 11:00:00.673111 21852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:00.678824 21852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:00.683128 21852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:00.683430 21852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:00.701119 21852 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:41159
I20260925 11:00:00.701139 21918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:41159 every 8 connection(s)
I20260925 11:00:00.702570 21852 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 11:00:00.703800 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 21852
I20260925 11:00:00.703955 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 11:00:00.704380 31994 external_mini_cluster.cc:1428] Setting key 33cda2a3d032f62417dd8742c8985ab5
I20260925 11:00:00.707276 21919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:00.713328 21922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:00.706333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36193 (local address 127.31.62.190:41159)
0925 11:00:00.707154 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:00.707170 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:00.707222 (+ 52us) server_negotiation.cc:407] Connection header received
0925 11:00:00.709494 (+ 2272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:00.709522 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:00.709954 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:00.710422 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:00.711383 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:00.711391 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:00.711407 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:00:00.711423 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:00.711520 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:00:00.711546 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:00.711747 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:00.711880 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 11:00:00.712063 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":327,"threads_started":1}
I20260925 11:00:00.717348 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:00:00.719714 21919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c: Bootstrap starting.
I20260925 11:00:00.723345 21919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:00.723812 21922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:00.721501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49595 (local address 127.31.62.190:41159)
0925 11:00:00.721671 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:00.721677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:00.722055 (+ 378us) server_negotiation.cc:407] Connection header received
0925 11:00:00.722231 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:00.722236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:00.722283 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:00.722361 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:00.723208 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:00.723214 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:00.723219 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:00:00.723237 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:00.723277 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:00:00.723292 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:00.723390 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:00.723486 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 11:00:00.723547 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 11:00:00.724555 21919 log.cc:824] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:00.727332 21919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c: No bootstrap required, opened a new log
I20260925 11:00:00.734517 21919 raft_consensus.cc:348] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61c9f72d16245ad97d041740a4ab44c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41159 } }
I20260925 11:00:00.734897 21919 raft_consensus.cc:374] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:00.734988 21919 raft_consensus.cc:729] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d61c9f72d16245ad97d041740a4ab44c, State: Initialized, Role: FOLLOWER
I20260925 11:00:00.735527 21919 consensus_queue.cc:260] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61c9f72d16245ad97d041740a4ab44c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41159 } }
I20260925 11:00:00.735750 21919 raft_consensus.cc:388] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:00:00.735894 21919 raft_consensus.cc:482] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:00:00.736047 21919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:00.738296 21919 raft_consensus.cc:504] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61c9f72d16245ad97d041740a4ab44c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41159 } }
I20260925 11:00:00.738772 21919 leader_election.cc:302] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d61c9f72d16245ad97d041740a4ab44c; no voters:
I20260925 11:00:00.739310 21919 leader_election.cc:288] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:00:00.739436 21929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:00.740029 21929 raft_consensus.cc:686] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [term 1 LEADER]: Becoming Leader. State: Replica: d61c9f72d16245ad97d041740a4ab44c, State: Running, Role: LEADER
I20260925 11:00:00.740527 21929 consensus_queue.cc:237] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61c9f72d16245ad97d041740a4ab44c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41159 } }
I20260925 11:00:00.741256 21919 sys_catalog.cc:564] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:00:00.743585 21931 sys_catalog.cc:455] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d61c9f72d16245ad97d041740a4ab44c. Latest consensus state: current_term: 1 leader_uuid: "d61c9f72d16245ad97d041740a4ab44c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61c9f72d16245ad97d041740a4ab44c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41159 } } }
I20260925 11:00:00.743554 21930 sys_catalog.cc:455] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d61c9f72d16245ad97d041740a4ab44c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61c9f72d16245ad97d041740a4ab44c" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 41159 } } }
I20260925 11:00:00.744073 21931 sys_catalog.cc:458] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:00.744074 21930 sys_catalog.cc:458] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:00.745610 21936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:00:00.751147 21936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:00:00.761067 21936 catalog_manager.cc:1347] Generated new cluster ID: bf8b2d156bd340ea8a506a722a7279f4
I20260925 11:00:00.761199 21936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:00:00.777532 21936 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:00:00.778064 21936 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:00:00.786615 21936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d61c9f72d16245ad97d041740a4ab44c: Generated new TSK 0
I20260925 11:00:00.787793 21936 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:00:00.805569 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 21852
W20260925 11:00:00.806741 21914 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36193
W20260925 11:00:00.807533 31994 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 21852: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:00:00.807657 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 21852
2026-09-25T11:00:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1775 ms)
[----------] 18 tests from AuthTokenIssuingTest (33049 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:00:00 dist-test-slave-nx7s krb5kdc[21950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:00:00 dist-test-slave-nx7s krb5kdc[21950](info): set up 2 sockets
Sep 25 11:00:00 dist-test-slave-nx7s krb5kdc[21950](info): commencing operation
krb5kdc: starting...
W20260925 11:00:01.829782 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.008s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 11:00:01 dist-test-slave-nx7s krb5kdc[21950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:00:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:00:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 11:00:02.092955 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:35913
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:40465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:35913,127.31.62.189:44389,127.31.62.188:42849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:02.426579 21963 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 11:00:02.426930 21963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:02.427028 21963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:02.427146 21963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:02.434556 21963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:02.434662 21963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:02.434723 21963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:02.434791 21963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:00:02.434875 21963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:02.445116 21963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:40465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:35913,127.31.62.189:44389,127.31.62.188:42849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:35913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:02.446475 21963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:02.447801 21963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:02.459091 21968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:02.459118 21969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:02.459201 21971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:02.460355 21963 server_base.cc:1034] running on GCE node
I20260925 11:00:02.461274 21963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:02.462739 21963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:02.463986 21963 hybrid_clock.cc:648] HybridClock initialized: now 1790334002463919 us; error 32 us; skew 500 ppm
I20260925 11:00:02.464388 21963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:02 dist-test-slave-nx7s krb5kdc[21950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334002, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:02.467769 21963 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 11:00:02.469416 21963 webserver.cc:460] Webserver started at http://127.31.62.190:33409/ using document root <none> and password file <none>
I20260925 11:00:02.470216 21963 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:02.470315 21963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:02.470633 21963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:02.473892 21963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "e7cbae9f831f48c68dcccd067018866e"
format_stamp: "Formatted at 2026-09-25 11:00:02 on dist-test-slave-nx7s"
server_key: "1cb0366f7b089b22fab30559fc2b75d4"
server_key_iv: "7eb24750d209a8ee486bd6c61e22d2ff"
server_key_version: "encryptionkey@0"
I20260925 11:00:02.474709 21963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "e7cbae9f831f48c68dcccd067018866e"
format_stamp: "Formatted at 2026-09-25 11:00:02 on dist-test-slave-nx7s"
server_key: "1cb0366f7b089b22fab30559fc2b75d4"
server_key_iv: "7eb24750d209a8ee486bd6c61e22d2ff"
server_key_version: "encryptionkey@0"
I20260925 11:00:02.480238 21963 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 11:00:02.484145 21978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:02.485759 21963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:00:02.485936 21963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "e7cbae9f831f48c68dcccd067018866e"
format_stamp: "Formatted at 2026-09-25 11:00:02 on dist-test-slave-nx7s"
server_key: "1cb0366f7b089b22fab30559fc2b75d4"
server_key_iv: "7eb24750d209a8ee486bd6c61e22d2ff"
server_key_version: "encryptionkey@0"
I20260925 11:00:02.486150 21963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:02.504769 21963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:02.509069 21963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:02.509326 21963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:02.526962 21963 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:35913
I20260925 11:00:02.526981 22029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:35913 every 8 connection(s)
I20260925 11:00:02.528373 21963 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 11:00:02.532307 22030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:02.537766 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 21963
I20260925 11:00:02.538021 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 11:00:02.538620 31994 external_mini_cluster.cc:1428] Setting key 369a1c455122b108d0992f73d6015ffe
I20260925 11:00:02.539026 22030 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:00:02 dist-test-slave-nx7s krb5kdc[21950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334002, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.62.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.189@KRBTEST.COM" created.
I20260925 11:00:02.563288 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:02.542474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:35913 (local address 127.0.0.1:34524)
0925 11:00:02.543457 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:02.543505 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:02.545821 (+ 2316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:02.546798 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:02.546806 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:02.547575 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:02.547944 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:02.547964 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:02.549690 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:02.549694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:02.550870 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:02.550884 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:02.551896 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:02.551904 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:02.552092 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:02.552781 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:02.552807 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:02.555136 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:02.558597 (+ 3461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:02.558610 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:02.558624 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:02.559057 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:02.559483 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:02.559489 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:02.559493 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:02.559676 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:02.560515 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:02.560524 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:02.560769 (+ 245us) client_negotiation.cc:769] Sending connection context
0925 11:00:02.561061 (+ 292us) client_negotiation.cc:240] Negotiation successful
0925 11:00:02.561832 (+ 771us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":129,"thread_start_us":462,"threads_started":1}
I20260925 11:00:02.563301 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:02.542474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34524 (local address 127.31.62.190:35913)
0925 11:00:02.544311 (+ 1837us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:02.544321 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:02.544390 (+ 69us) server_negotiation.cc:407] Connection header received
0925 11:00:02.546245 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:02.546252 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:02.546538 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:02.546785 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:02.548163 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:02.549523 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:02.551021 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:02.551645 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:02.551875 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:02.555375 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:02.555385 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:02.555398 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:02.555437 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:02.558458 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:02.559193 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:02.559201 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:02.559204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:02.559371 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:02.559798 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:02.559804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:02.559808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:02.560370 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:02.560584 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:02.561214 (+ 630us) server_negotiation.cc:299] Negotiation successful
0925 11:00:02.561632 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":1608,"thread_start_us":1391,"threads_started":1}
I20260925 11:00:02.566720 22030 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:00:02.567797 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:02.567372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:44389 (local address 127.0.0.1:35452)
0925 11:00:02.567543 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:02.567663 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260925 11:00:02.568061 21980 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
I20260925 11:00:02.569710 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:02.569323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:44389 (local address 127.0.0.1:35460)
0925 11:00:02.569480 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:02.569555 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 11:00:02.569957 22030 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.189:44389: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.189.keytab.
Entry for principal kudu/127.31.62.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.189.keytab.
I20260925 11:00:02.591770 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.189:44389
--webserver_interface=127.31.62.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.189.keytab
--principal=kudu/127.31.62.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:40465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:35913,127.31.62.189:44389,127.31.62.188:42849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:00:02.619323 22030 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } attempt: 1
I20260925 11:00:02.620544 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:02.620015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:44389 (local address 127.0.0.1:35476)
0925 11:00:02.620230 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:02.620380 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260925 11:00:02.621634 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:02.621218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:44389 (local address 127.0.0.1:35486)
0925 11:00:02.621371 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:02.621468 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 11:00:02.621830 22030 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.189:44389: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
I20260925 11:00:02.690061 22030 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } attempt: 2
I20260925 11:00:02.691329 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:02.690752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:44389 (local address 127.0.0.1:35492)
0925 11:00:02.691037 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:02.691147 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260925 11:00:02.692384 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:02.691959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:44389 (local address 127.0.0.1:35494)
0925 11:00:02.692121 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:02.692207 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260925 11:00:02.692559 22030 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.189:44389: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
I20260925 11:00:02.783851 22030 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } attempt: 3
I20260925 11:00:02.785188 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:02.784596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:44389 (local address 127.0.0.1:35498)
0925 11:00:02.784874 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:02.785001 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260925 11:00:02.786425 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:02.785927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:44389 (local address 127.0.0.1:35502)
0925 11:00:02.786109 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:02.786220 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260925 11:00:02.786645 22030 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.189:44389: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
W20260925 11:00:02.917356 22037 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 11:00:02.917680 22037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:02.917789 22037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:02.917901 22037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:02.925236 22037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:02.925345 22037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:02.925433 22037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:02.925494 22037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:00:02.925575 22037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:02.929913 22030 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } attempt: 4
I20260925 11:00:02.931180 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:02.930579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:44389 (local address 127.0.0.1:35504)
0925 11:00:02.930905 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:02.931014 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260925 11:00:02.932264 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:02.931870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:44389 (local address 127.0.0.1:35518)
0925 11:00:02.932015 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:02.932114 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260925 11:00:02.932493 22030 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.189:44389: Network error: Client connection negotiation failed: client connection to 127.31.62.189:44389: connect: Connection refused (error 111)
I20260925 11:00:02.936425 22037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:40465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:35913,127.31.62.189:44389,127.31.62.188:42849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.189.keytab
--principal=kudu/127.31.62.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.189:44389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.62.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:02.937740 22037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:02.939150 22037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:02.950408 22042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:02.951290 22043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:02.951699 22045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:02.952504 22037 server_base.cc:1034] running on GCE node
I20260925 11:00:02.953421 22037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:02.954869 22037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:02.956086 22037 hybrid_clock.cc:648] HybridClock initialized: now 1790334002956014 us; error 41 us; skew 500 ppm
I20260925 11:00:02.956409 22037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:02 dist-test-slave-nx7s krb5kdc[21950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334002, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:02.959718 22037 init.cc:377] Logged in from keytab as kudu/127.31.62.189@KRBTEST.COM (short username kudu)
I20260925 11:00:02.961347 22037 webserver.cc:460] Webserver started at http://127.31.62.189:38007/ using document root <none> and password file <none>
I20260925 11:00:02.962091 22037 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:02.962225 22037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:02.962545 22037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:02.965581 22037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/data/instance:
uuid: "664b68ba24784247b0265fd948db57f1"
format_stamp: "Formatted at 2026-09-25 11:00:02 on dist-test-slave-nx7s"
server_key: "ac003fd85026b272868e40eb7e66878d"
server_key_iv: "95d09a160dc9d9c968e69843874caacb"
server_key_version: "encryptionkey@0"
I20260925 11:00:02.966354 22037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/wal/instance:
uuid: "664b68ba24784247b0265fd948db57f1"
format_stamp: "Formatted at 2026-09-25 11:00:02 on dist-test-slave-nx7s"
server_key: "ac003fd85026b272868e40eb7e66878d"
server_key_iv: "95d09a160dc9d9c968e69843874caacb"
server_key_version: "encryptionkey@0"
I20260925 11:00:02.971481 22037 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:00:02.975133 22052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:02.976562 22037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:00:02.976722 22037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/wal
uuid: "664b68ba24784247b0265fd948db57f1"
format_stamp: "Formatted at 2026-09-25 11:00:02 on dist-test-slave-nx7s"
server_key: "ac003fd85026b272868e40eb7e66878d"
server_key_iv: "95d09a160dc9d9c968e69843874caacb"
server_key_version: "encryptionkey@0"
I20260925 11:00:02.976966 22037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:03.006953 22037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:03.011489 22037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:03.011775 22037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:03.029331 22037 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.189:44389
I20260925 11:00:03.029349 22103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.189:44389 every 8 connection(s)
I20260925 11:00:03.030687 22037 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/data/info.pb
I20260925 11:00:03.034364 22104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:03.036145 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 22037
I20260925 11:00:03.036301 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-1/wal/instance
I20260925 11:00:03.036697 31994 external_mini_cluster.cc:1428] Setting key 862a15f27a0c9858aca46ac1544cada7
I20260925 11:00:03.041056 22104 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:00:03 dist-test-slave-nx7s krb5kdc[21950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334002, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.189@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.62.188@KRBTEST.COM; defaulting to no policy
I20260925 11:00:03.061821 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.043563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34536 (local address 127.31.62.190:35913)
0925 11:00:03.043770 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:03.043777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:03.045105 (+ 1328us) server_negotiation.cc:407] Connection header received
0925 11:00:03.047106 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:03.047113 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:03.047195 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:03.047300 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:03.048814 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.049730 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.051277 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.051829 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.051938 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.055673 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:03.055679 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:03.055682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:03.055720 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:03.058534 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.059458 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.059463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.059465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.059607 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.060071 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.060075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.060077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.060417 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:03.060532 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:03.061327 (+ 795us) server_negotiation.cc:299] Negotiation successful
0925 11:00:03.061535 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 11:00:03.062809 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.044094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:35913 (local address 127.0.0.1:34536)
0925 11:00:03.044917 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.044984 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:03.046825 (+ 1841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:03.047498 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:03.047507 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:03.048066 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:03.048646 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.048663 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.049889 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.049894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.051133 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.051153 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.052063 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.052068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.052273 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.052939 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:03.052970 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:03.055419 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:03.058681 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.058690 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.058710 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.059327 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.059718 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.059723 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.059726 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.059954 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.060546 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:03.060553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:03.060953 (+ 400us) client_negotiation.cc:769] Sending connection context
0925 11:00:03.061197 (+ 244us) client_negotiation.cc:240] Negotiation successful
0925 11:00:03.061650 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":307,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.188@KRBTEST.COM" created.
I20260925 11:00:03.064680 22104 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:00:03 dist-test-slave-nx7s krb5kdc[21950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334002, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.189@KRBTEST.COM for kudu/127.31.62.189@KRBTEST.COM
I20260925 11:00:03.080219 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.065316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:44389 (local address 127.0.0.1:35520)
0925 11:00:03.065576 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.065594 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:03.065693 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:03.066566 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:03.066571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:03.066985 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:03.067145 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.067154 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.068617 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.068622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.069813 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.069828 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.070703 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.070708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.071018 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.071631 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:03.071653 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:03.073784 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:03.077826 (+ 4042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.077830 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.077833 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.078194 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.078599 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.078602 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.078604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.078737 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.079455 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:03.079460 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:03.079572 (+ 112us) client_negotiation.cc:769] Sending connection context
0925 11:00:03.079680 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 11:00:03.079875 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":107}
I20260925 11:00:03.080603 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.065569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35520 (local address 127.31.62.189:44389)
0925 11:00:03.066164 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:03.066170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:03.066193 (+ 23us) server_negotiation.cc:407] Connection header received
0925 11:00:03.066232 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:03.066238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:03.066419 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:03.066527 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:03.067339 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.068469 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.069970 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.070555 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.070692 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.074025 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:03.074042 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:03.074060 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:03.074100 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:03.077673 (+ 3573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.078331 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.078341 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.078345 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.078503 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.078845 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.078851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.078854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.079339 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:03.079530 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:03.079831 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 11:00:03.080097 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":449,"thread_start_us":317,"threads_started":1}
I20260925 11:00:03.083140 22104 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:00:03.084299 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.083820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:49906)
0925 11:00:03.084050 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.084156 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260925 11:00:03.084585 22056 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
I20260925 11:00:03.086236 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.085911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:49920)
0925 11:00:03.086055 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.086130 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260925 11:00:03.086453 22104 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:42849: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.188.keytab.
Entry for principal kudu/127.31.62.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.188.keytab.
I20260925 11:00:03.097882 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.188:42849
--webserver_interface=127.31.62.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.188.keytab
--principal=kudu/127.31.62.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:40465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:35913,127.31.62.189:44389,127.31.62.188:42849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:00:03.135752 22104 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } attempt: 1
I20260925 11:00:03.137022 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.136527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:49928)
0925 11:00:03.136715 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.136819 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260925 11:00:03.137913 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.137626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:49944)
0925 11:00:03.137740 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.137813 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260925 11:00:03.138191 22104 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:42849: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
I20260925 11:00:03.206390 22104 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } attempt: 2
I20260925 11:00:03.207523 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.207071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:49954)
0925 11:00:03.207259 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.207351 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260925 11:00:03.208666 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.208246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:49962)
0925 11:00:03.208394 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.208505 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 11:00:03.209056 22104 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:42849: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
I20260925 11:00:03.231781 22030 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } attempt: 5
Sep 25 11:00:03 dist-test-slave-nx7s krb5kdc[21950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334002, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for kudu/127.31.62.189@KRBTEST.COM
I20260925 11:00:03.246642 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.232793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35522 (local address 127.31.62.189:44389)
0925 11:00:03.232990 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:03.232994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:03.233013 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:00:03.233122 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:03.233126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:03.233185 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:03.233266 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:03.234375 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.235086 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.236715 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.237234 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.237330 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.241096 (+ 3766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:03.241101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:03.241104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:03.241140 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:03.243674 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.244614 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.244619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.244622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.244771 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.245249 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.245252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.245254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.245584 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:03.245700 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:03.246156 (+ 456us) server_negotiation.cc:299] Negotiation successful
0925 11:00:03.246381 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 11:00:03.246707 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.232546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:44389 (local address 127.0.0.1:35522)
0925 11:00:03.232788 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.232815 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:03.232995 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:03.233430 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:03.233436 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:03.233948 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:03.234179 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.234192 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.235228 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.235233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.236554 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.236570 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.237392 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.237398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.237598 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.238252 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:03.238275 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:03.240835 (+ 2560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:03.243840 (+ 3005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.243847 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.243852 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.244431 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.244935 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.244941 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.244945 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.245146 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.245704 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:03.245708 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:03.245909 (+ 201us) client_negotiation.cc:769] Sending connection context
0925 11:00:03.246064 (+ 155us) client_negotiation.cc:240] Negotiation successful
0925 11:00:03.246365 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260925 11:00:03.247714 22030 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:00:03.248988 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.248477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:49966)
0925 11:00:03.248666 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.248779 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 11:00:03.249955 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.249576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:49976)
0925 11:00:03.249712 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.249827 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 11:00:03.250159 22030 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:42849: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
I20260925 11:00:03.300333 22104 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } attempt: 3
I20260925 11:00:03.301406 22030 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } attempt: 1
I20260925 11:00:03.301611 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.301015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:49986)
0925 11:00:03.301254 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.301413 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260925 11:00:03.302474 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.302038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:49992)
0925 11:00:03.302220 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.302308 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 11:00:03.302902 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.302369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:50004)
0925 11:00:03.302634 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.302737 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
W20260925 11:00:03.303087 22104 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:42849: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
I20260925 11:00:03.303606 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.303265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:50008)
0925 11:00:03.303419 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.303496 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 11:00:03.303812 22030 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:42849: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
I20260925 11:00:03.372040 22030 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } attempt: 2
I20260925 11:00:03.373230 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.372690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:50018)
0925 11:00:03.372940 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.373059 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260925 11:00:03.374157 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.373828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:50032)
0925 11:00:03.373977 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.374046 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260925 11:00:03.374348 22030 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:42849: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
W20260925 11:00:03.439224 22111 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 11:00:03.439621 22111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:03.439718 22111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:03.439844 22111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 11:00:03.446352 22104 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } attempt: 4
W20260925 11:00:03.447479 22111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:03.447582 22111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:03.447672 22111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:03.447733 22111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260925 11:00:03.447788 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.447185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:50044)
0925 11:00:03.447491 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.447605 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260925 11:00:03.447817 22111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:03.448779 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.448470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:50052)
0925 11:00:03.448595 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.448670 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 11:00:03.449014 22104 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:42849: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
I20260925 11:00:03.458550 22111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:40465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:35913,127.31.62.189:44389,127.31.62.188:42849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.188.keytab
--principal=kudu/127.31.62.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.188:42849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.62.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:03.459894 22111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:03.461328 22111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:03.473508 22116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:03.473730 22119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:03.473476 22117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:03.474954 22111 server_base.cc:1034] running on GCE node
I20260925 11:00:03.475862 22111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:03.477485 22111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:03.478714 22111 hybrid_clock.cc:648] HybridClock initialized: now 1790334003478621 us; error 76 us; skew 500 ppm
I20260925 11:00:03.479071 22111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:00:03.480530 22030 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } attempt: 3
I20260925 11:00:03.481679 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.481191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:50056)
0925 11:00:03.481398 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.481513 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
Sep 25 11:00:03 dist-test-slave-nx7s krb5kdc[21950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334003, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:03.482765 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.482315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:50064)
0925 11:00:03.482523 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.482616 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260925 11:00:03.482728 22111 init.cc:377] Logged in from keytab as kudu/127.31.62.188@KRBTEST.COM (short username kudu)
W20260925 11:00:03.483008 22030 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:42849: Network error: Client connection negotiation failed: client connection to 127.31.62.188:42849: connect: Connection refused (error 111)
I20260925 11:00:03.484402 22111 webserver.cc:460] Webserver started at http://127.31.62.188:32979/ using document root <none> and password file <none>
I20260925 11:00:03.485172 22111 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:03.485261 22111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:03.485556 22111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:03.488592 22111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/data/instance:
uuid: "5dd0424f0b5f48bba832b7423523222b"
format_stamp: "Formatted at 2026-09-25 11:00:03 on dist-test-slave-nx7s"
server_key: "ddfbb83e546e5eb4b46f5553600c7705"
server_key_iv: "3b72d6c303c2a4bfb9747ce25003c9f0"
server_key_version: "encryptionkey@0"
I20260925 11:00:03.489490 22111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/wal/instance:
uuid: "5dd0424f0b5f48bba832b7423523222b"
format_stamp: "Formatted at 2026-09-25 11:00:03 on dist-test-slave-nx7s"
server_key: "ddfbb83e546e5eb4b46f5553600c7705"
server_key_iv: "3b72d6c303c2a4bfb9747ce25003c9f0"
server_key_version: "encryptionkey@0"
I20260925 11:00:03.495805 22111 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260925 11:00:03.501031 22126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:03.503059 22111 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 11:00:03.503284 22111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/wal
uuid: "5dd0424f0b5f48bba832b7423523222b"
format_stamp: "Formatted at 2026-09-25 11:00:03 on dist-test-slave-nx7s"
server_key: "ddfbb83e546e5eb4b46f5553600c7705"
server_key_iv: "3b72d6c303c2a4bfb9747ce25003c9f0"
server_key_version: "encryptionkey@0"
I20260925 11:00:03.503566 22111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:03.519152 22111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:03.523340 22111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:03.523567 22111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:03.542618 22111 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.188:42849
I20260925 11:00:03.542637 22177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.188:42849 every 8 connection(s)
I20260925 11:00:03.543975 22111 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/data/info.pb
I20260925 11:00:03.547775 22178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:03.552146 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 22111
I20260925 11:00:03.552294 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790333647382755-31994-0/minicluster-data/master-2/wal/instance
I20260925 11:00:03.552712 31994 external_mini_cluster.cc:1428] Setting key f7d192147e44749e9e457f794a265d2f
I20260925 11:00:03.554757 22178 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:00:03 dist-test-slave-nx7s krb5kdc[21950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334003, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.188@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
Sep 25 11:00:03 dist-test-slave-nx7s krb5kdc[21950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.188@KRBTEST.COM
I20260925 11:00:03.574338 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.557880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34542 (local address 127.31.62.190:35913)
0925 11:00:03.558086 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:03.558092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:03.558564 (+ 472us) server_negotiation.cc:407] Connection header received
0925 11:00:03.558731 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:03.558737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:03.558816 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:03.558921 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:03.560094 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.560780 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.562799 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.563468 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.563611 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.567569 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:03.567575 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:03.567578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:03.567615 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:03.570684 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.571497 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.571503 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.571507 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.571642 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.572091 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.572094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.572096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.572436 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:03.572554 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:03.573486 (+ 932us) server_negotiation.cc:299] Negotiation successful
0925 11:00:03.573706 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 11:00:03.575186 22182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.557719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:35913 (local address 127.0.0.1:34542)
0925 11:00:03.558413 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.558454 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:03.558609 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:03.559002 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:03.559008 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:03.559585 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:03.559931 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.559949 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.560968 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.560973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.562636 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.562653 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.563684 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.563689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.563896 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.564652 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:03.564683 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:03.567331 (+ 2648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:03.570813 (+ 3482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.570823 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.570833 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.571379 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.571745 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.571749 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.571752 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.571989 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.572556 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:03.572566 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:03.573111 (+ 545us) client_negotiation.cc:769] Sending connection context
0925 11:00:03.573388 (+ 277us) client_negotiation.cc:240] Negotiation successful
0925 11:00:03.573966 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":41,"thread_start_us":309,"threads_started":1}
I20260925 11:00:03.575204 22181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.554681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50068 (local address 127.31.62.188:42849)
0925 11:00:03.555506 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:03.555521 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:03.555571 (+ 50us) server_negotiation.cc:407] Connection header received
0925 11:00:03.557754 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:03.557779 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:03.558310 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:03.558650 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:03.560082 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.561249 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.562906 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.563756 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.563900 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.568171 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:03.568180 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:03.568194 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:03.568223 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:03.571392 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.572222 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.572232 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.572235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.572375 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.572775 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.572778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.572780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.573188 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:03.573471 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:03.573781 (+ 310us) server_negotiation.cc:299] Negotiation successful
0925 11:00:03.574095 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":569,"thread_start_us":353,"threads_started":1}
I20260925 11:00:03.577251 22178 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:00:03 dist-test-slave-nx7s krb5kdc[21950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334003, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.188@KRBTEST.COM for kudu/127.31.62.189@KRBTEST.COM
Sep 25 11:00:03 dist-test-slave-nx7s krb5kdc[21950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 11:00:03.591343 22182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.578063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:44389 (local address 127.0.0.1:35534)
0925 11:00:03.578233 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.578250 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:03.578380 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:03.578740 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:03.578745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:03.579189 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:03.579445 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.579468 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.580767 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.580771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.582006 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.582020 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.582694 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.582700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.582883 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.583423 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:03.583441 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:03.585956 (+ 2515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:03.589068 (+ 3112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.589074 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.589078 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.589483 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.589879 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.589885 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.589889 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.590035 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.590541 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:03.590546 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:03.590691 (+ 145us) client_negotiation.cc:769] Sending connection context
0925 11:00:03.590855 (+ 164us) client_negotiation.cc:240] Negotiation successful
0925 11:00:03.591085 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 11:00:03.591554 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.578134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35534 (local address 127.31.62.189:44389)
0925 11:00:03.578357 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:03.578363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:03.578382 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:00:03.578506 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:03.578512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:03.578581 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:03.578676 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:03.579685 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.580632 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.582133 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.582560 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.582654 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.586168 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:03.586173 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:03.586176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:03.586211 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:03.588927 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.589596 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.589602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.589604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.589774 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.590144 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.590149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.590152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.590427 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:03.590581 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:03.590972 (+ 391us) server_negotiation.cc:299] Negotiation successful
0925 11:00:03.591196 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260925 11:00:03.592401 22178 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:00:03.594444 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.578533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34550 (local address 127.31.62.190:35913)
0925 11:00:03.578708 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:03.578713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:03.578789 (+ 76us) server_negotiation.cc:407] Connection header received
0925 11:00:03.578947 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:03.578950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:03.579002 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:03.579069 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:03.580529 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.581445 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.582826 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.583227 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.583346 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.588345 (+ 4999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:03.588351 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:03.588354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:03.588389 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:03.591214 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.592174 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.592180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.592183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.592341 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.592704 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.592709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.592712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.593069 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:03.593328 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:03.593889 (+ 561us) server_negotiation.cc:299] Negotiation successful
0925 11:00:03.594120 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Sep 25 11:00:03 dist-test-slave-nx7s krb5kdc[21950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334003, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.188@KRBTEST.COM for kudu/127.31.62.188@KRBTEST.COM
Sep 25 11:00:03 dist-test-slave-nx7s krb5kdc[21950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.189@KRBTEST.COM
I20260925 11:00:03.605580 22182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.593346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:50080)
0925 11:00:03.593509 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.593527 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:03.593625 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:03.594038 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:03.594043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:03.594473 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:03.594641 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.594652 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.595724 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.595729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.596994 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.597007 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.597688 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.597694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.597924 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.598457 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:03.598472 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:03.600683 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:03.603373 (+ 2690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.603379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.603383 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.603730 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.604067 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.604072 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.604076 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.604195 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.604691 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:03.604694 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:03.604826 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 11:00:03.604990 (+ 164us) client_negotiation.cc:240] Negotiation successful
0925 11:00:03.605218 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 11:00:03.605674 22181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.593346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50080 (local address 127.31.62.188:42849)
0925 11:00:03.593510 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:03.593516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:03.593568 (+ 52us) server_negotiation.cc:407] Connection header received
0925 11:00:03.593728 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:03.593734 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:03.593795 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:03.593884 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:03.594762 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.595617 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.597119 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.597540 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.597639 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.600894 (+ 3255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:03.600900 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:03.600903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:03.600932 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:03.603235 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.603838 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.603843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.603847 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.603972 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.604283 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.604288 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.604292 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.604574 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:03.604704 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:03.605095 (+ 391us) server_negotiation.cc:299] Negotiation successful
0925 11:00:03.605396 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":51}
I20260925 11:00:03.610230 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.596152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35536 (local address 127.31.62.189:44389)
0925 11:00:03.596296 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:03.596300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:03.596313 (+ 13us) server_negotiation.cc:407] Connection header received
0925 11:00:03.596357 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:03.596360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:03.596412 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:03.596487 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:03.597917 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.598739 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.600224 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.600655 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.600763 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.604715 (+ 3952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:03.604721 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:03.604726 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:03.604764 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:03.607532 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.608340 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.608344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.608347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.608471 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.608934 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.608939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.608942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.609199 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:03.609450 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:03.609721 (+ 271us) server_negotiation.cc:299] Negotiation successful
0925 11:00:03.609939 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 11:00:03.611217 22178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b: Bootstrap starting.
I20260925 11:00:03.614825 22178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:03.615011 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:00:03.615893 22178 log.cc:824] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:03.618503 22178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b: No bootstrap required, opened a new log
I20260925 11:00:03.617501 22005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e7cbae9f831f48c68dcccd067018866e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34550
I20260925 11:00:03.625376 22178 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } }
I20260925 11:00:03.625694 22178 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd0424f0b5f48bba832b7423523222b, State: Initialized, Role: FOLLOWER
I20260925 11:00:03.626121 22178 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } }
I20260925 11:00:03.627004 22184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } } }
I20260925 11:00:03.627329 22184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:00:03.628155 22178 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b [sys.catalog]: configured and running, proceeding with master startup.
W20260925 11:00:03.637058 22195 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 11:00:03.637193 22195 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 11:00:03.635169 22111 master_runner.cc:186] Error getting master registration for 127.31.62.190:35913: OK, instance_id { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" instance_seqno: 1790334002521923 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 11:00:03.646072 22111 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 11:00:03.660235 22030 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } attempt: 4
Sep 25 11:00:03 dist-test-slave-nx7s krb5kdc[21950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334002, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for kudu/127.31.62.188@KRBTEST.COM
I20260925 11:00:03.676595 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.661079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:50086)
0925 11:00:03.661368 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.661388 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:03.661532 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:03.661992 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:03.661997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:03.662501 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:03.662721 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.662734 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.664189 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.664195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.665720 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.665737 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.666715 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.666722 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.666939 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.667622 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:03.667647 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:03.670290 (+ 2643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:03.673988 (+ 3698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.673994 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.673998 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.674371 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.674754 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.674759 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.674763 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.674907 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.675611 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:03.675616 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:03.675787 (+ 171us) client_negotiation.cc:769] Sending connection context
0925 11:00:03.675911 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 11:00:03.676199 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260925 11:00:03.676618 22181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.661067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50086 (local address 127.31.62.188:42849)
0925 11:00:03.661249 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:03.661255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:03.661465 (+ 210us) server_negotiation.cc:407] Connection header received
0925 11:00:03.661671 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:03.661677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:03.661759 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:03.661926 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:03.662913 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.664027 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.665867 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.666560 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.666671 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.670465 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:03.670469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:03.670472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:03.670502 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:03.673855 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.674502 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.674508 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.674512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.674654 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.675019 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.675025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.675028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.675502 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:03.675652 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:03.676045 (+ 393us) server_negotiation.cc:299] Negotiation successful
0925 11:00:03.676269 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 11:00:03.682971 22030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e: Bootstrap starting.
I20260925 11:00:03.687314 22030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:03.688737 22030 log.cc:824] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:03.692416 22030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e: No bootstrap required, opened a new log
I20260925 11:00:03.700227 22030 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } }
I20260925 11:00:03.700698 22030 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7cbae9f831f48c68dcccd067018866e, State: Initialized, Role: FOLLOWER
I20260925 11:00:03.701282 22030 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } }
I20260925 11:00:03.702319 22198 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } } }
I20260925 11:00:03.702684 22198 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:00:03.703626 22030 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:00:03.703712 22005 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 11:00:03.703912 22005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:03.706664 22005 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } }
I20260925 11:00:03.709164 22153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7cbae9f831f48c68dcccd067018866e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5dd0424f0b5f48bba832b7423523222b"
I20260925 11:00:03.709650 22153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:03.713186 22153 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7cbae9f831f48c68dcccd067018866e in term 1.
I20260925 11:00:03.713598 22005 leader_election.cc:288] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [CANDIDATE]: Term 1 election: Requested vote from peers 664b68ba24784247b0265fd948db57f1 (127.31.62.189:44389), 5dd0424f0b5f48bba832b7423523222b (127.31.62.188:42849)
I20260925 11:00:03.713924 21982 leader_election.cc:302] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dd0424f0b5f48bba832b7423523222b, e7cbae9f831f48c68dcccd067018866e; no voters:
I20260925 11:00:03.714690 22198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:03.716497 22198 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [term 1 LEADER]: Becoming Leader. State: Replica: e7cbae9f831f48c68dcccd067018866e, State: Running, Role: LEADER
I20260925 11:00:03.717078 22198 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } }
I20260925 11:00:03.728802 22204 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7cbae9f831f48c68dcccd067018866e. Latest consensus state: current_term: 1 leader_uuid: "e7cbae9f831f48c68dcccd067018866e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } } }
I20260925 11:00:03.729413 22204 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:03.734876 22211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:00:03.744395 22211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:00:03.748205 22104 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } attempt: 5
Sep 25 11:00:03 dist-test-slave-nx7s krb5kdc[21950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334002, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.189@KRBTEST.COM for kudu/127.31.62.188@KRBTEST.COM
W20260925 11:00:03.762363 21984 tablet.cc:2369] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:00:03.766460 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.748834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:42849 (local address 127.0.0.1:50088)
0925 11:00:03.749249 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:03.749268 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:03.749388 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:03.750281 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:03.750287 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:03.750797 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:03.751012 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.751026 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.752645 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.752650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.754134 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:03.754151 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.755326 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.755333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:03.755568 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.756240 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:03.756265 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:03.759139 (+ 2874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:03.763513 (+ 4374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.763520 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.763524 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.764089 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.764572 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:03.764577 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:03.764581 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:03.764746 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:03.765451 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:03.765456 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:03.765610 (+ 154us) client_negotiation.cc:769] Sending connection context
0925 11:00:03.765766 (+ 156us) client_negotiation.cc:240] Negotiation successful
0925 11:00:03.766081 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268}
I20260925 11:00:03.767419 22181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.749604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50088 (local address 127.31.62.188:42849)
0925 11:00:03.749781 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:03.749786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:03.749806 (+ 20us) server_negotiation.cc:407] Connection header received
0925 11:00:03.749866 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:03.749872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:03.749981 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:03.750086 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:03.751210 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.752470 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.754308 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.755011 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.755128 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.759387 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:03.759393 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:03.759398 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:03.759440 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:03.763317 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.764241 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.764248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.764252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.764422 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.764895 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.764900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.764904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.765255 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:03.765392 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:03.766827 (+ 1435us) server_negotiation.cc:299] Negotiation successful
0925 11:00:03.767063 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 11:00:03.773283 22104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1: Bootstrap starting.
W20260925 11:00:03.775626 22132 tablet.cc:2369] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:00:03.778424 22104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:03.780040 22104 log.cc:824] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:03.784034 22153 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b [term 1 FOLLOWER]: Refusing update from remote peer e7cbae9f831f48c68dcccd067018866e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:03.785229 22204 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:03.839897 22181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.752772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50102 (local address 127.31.62.188:42849)
0925 11:00:03.777154 (+ 24382us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:03.777160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:03.777181 (+ 21us) server_negotiation.cc:407] Connection header received
0925 11:00:03.777243 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:03.777250 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:03.777329 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:03.777439 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:03.797277 (+ 19838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.798364 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.802640 (+ 4276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.803721 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.803842 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.815595 (+ 11753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:03.815605 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:03.815610 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:03.815666 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:03.819756 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.832272 (+ 12516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.832281 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.832285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.832524 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.835023 (+ 2499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.835029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.835033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.835499 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:03.835780 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:03.839144 (+ 3364us) server_negotiation.cc:299] Negotiation successful
0925 11:00:03.839458 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24252,"thread_start_us":28207,"threads_started":1}
I20260925 11:00:03.785926 21963 master_runner.cc:186] Error getting master registration for 127.31.62.189:44389: OK, instance_id { permanent_uuid: "664b68ba24784247b0265fd948db57f1" instance_seqno: 1790334003024365 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 11:00:03.844321 21963 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 11:00:03.845312 22104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1: No bootstrap required, opened a new log
I20260925 11:00:03.861869 22184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7cbae9f831f48c68dcccd067018866e. Latest consensus state: current_term: 1 leader_uuid: "e7cbae9f831f48c68dcccd067018866e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } } }
I20260925 11:00:03.862180 22184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:00:03.864590 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.777117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35542 (local address 127.31.62.189:44389)
0925 11:00:03.777284 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:03.777289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:03.779982 (+ 2693us) server_negotiation.cc:407] Connection header received
0925 11:00:03.780341 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:03.780347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:03.780430 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:03.780502 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:03.807713 (+ 27211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.808958 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.829123 (+ 20165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.829776 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.829911 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.857312 (+ 27401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:03.857321 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:03.857326 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:03.857378 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:03.860982 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.861939 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.861946 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.861949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.862126 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.862647 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.862653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.862656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.863085 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:03.863325 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:03.863950 (+ 625us) server_negotiation.cc:299] Negotiation successful
0925 11:00:03.864207 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:00:03.864965 22219 mvcc.cc:205] Tried to move back new op lower bound from 7333208079486464000 to 7333208079230996480. Current Snapshot: MvccSnapshot[applied={T|T < 7333208079486464000}]
I20260925 11:00:03.869585 22184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7cbae9f831f48c68dcccd067018866e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } } }
I20260925 11:00:03.869874 22184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:00:03.871838 22224 mvcc.cc:205] Tried to move back new op lower bound from 7333208079486464000 to 7333208079230996480. Current Snapshot: MvccSnapshot[applied={T|T < 7333208079486464000}]
I20260925 11:00:03.879344 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:03.756879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34564 (local address 127.31.62.190:35913)
0925 11:00:03.757093 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:03.757099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:03.774080 (+ 16981us) server_negotiation.cc:407] Connection header received
0925 11:00:03.777910 (+ 3830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:03.777917 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:03.777998 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:03.778064 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:03.827145 (+ 49081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.828279 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.851776 (+ 23497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:03.852514 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:03.852658 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:03.865415 (+ 12757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:03.865424 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:03.865429 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:03.865478 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:03.874990 (+ 9512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.875993 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.876001 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.876005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.876180 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:03.876670 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:03.876675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:03.876679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:03.877121 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:03.877390 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:03.878642 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0925 11:00:03.878919 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260925 11:00:03.879973 22198 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7cbae9f831f48c68dcccd067018866e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } } }
I20260925 11:00:03.880317 22198 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:03.886750 22211 catalog_manager.cc:1347] Generated new cluster ID: fdbaeba6a2b3470192db4aa1f5b443e5
I20260925 11:00:03.886859 22211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:00:03.886793 22104 raft_consensus.cc:348] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } }
I20260925 11:00:03.887252 22104 raft_consensus.cc:729] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 664b68ba24784247b0265fd948db57f1, State: Initialized, Role: FOLLOWER
I20260925 11:00:03.887812 22104 consensus_queue.cc:260] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } }
I20260925 11:00:03.905606 22104 sys_catalog.cc:564] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:00:03.906071 22079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7cbae9f831f48c68dcccd067018866e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "664b68ba24784247b0265fd948db57f1"
I20260925 11:00:03.907733 22078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:03.912039 22078 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1 [term 1 FOLLOWER]: Refusing update from remote peer e7cbae9f831f48c68dcccd067018866e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:03.920416 22211 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:00:03.922359 22211 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:00:03.923170 22228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } } }
I20260925 11:00:03.923568 22228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:00:03.926393 22079 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7cbae9f831f48c68dcccd067018866e in term 1.
I20260925 11:00:03.927896 22198 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [LEADER]: Connected to new peer: Peer: permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:03.953024 22211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e: Generated new TSK 0
I20260925 11:00:03.954234 22211 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:00:04.045686 22228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7cbae9f831f48c68dcccd067018866e. Latest consensus state: current_term: 1 leader_uuid: "e7cbae9f831f48c68dcccd067018866e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } } }
I20260925 11:00:04.046032 22228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:00:04.047443 22204 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7cbae9f831f48c68dcccd067018866e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } } }
I20260925 11:00:04.047739 22204 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7cbae9f831f48c68dcccd067018866e [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:04.049979 22234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7cbae9f831f48c68dcccd067018866e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7cbae9f831f48c68dcccd067018866e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 35913 } } peers { permanent_uuid: "664b68ba24784247b0265fd948db57f1" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 44389 } } peers { permanent_uuid: "5dd0424f0b5f48bba832b7423523222b" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 42849 } } }
I20260925 11:00:04.050421 22234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 11:00:04.083640 31994 mini_kdc.cc:348] Time spent kdestroy: real 0.116s user 0.002s sys 0.023s
I20260925 11:00:04.086890 22245 catalog_manager.cc:1259] Loaded cluster ID: fdbaeba6a2b3470192db4aa1f5b443e5
I20260925 11:00:04.087005 22245 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1: loading cluster ID for follower catalog manager: success
I20260925 11:00:04.092471 22245 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1: acquiring CA information for follower catalog manager: success
I20260925 11:00:04.106835 22245 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 664b68ba24784247b0265fd948db57f1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 11:00:04.140475 22181 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:04.131255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50114 (local address 127.31.62.188:42849)
0925 11:00:04.131480 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.131487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.137272 (+ 5785us) server_negotiation.cc:407] Connection header received
0925 11:00:04.137448 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.137454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.137536 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.137637 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:04.140168 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50114: BlockingRecv error: recv got EOF from 127.0.0.1:50114 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260925 11:00:04.143522 22108 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:04.138036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35554 (local address 127.31.62.189:44389)
0925 11:00:04.138241 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.138247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.141309 (+ 3062us) server_negotiation.cc:407] Connection header received
0925 11:00:04.142154 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.142161 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.142242 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.142317 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:04.143290 (+ 973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35554: BlockingRecv error: recv got EOF from 127.0.0.1:35554 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260925 11:00:04.145097 22033 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:04.122814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34572 (local address 127.31.62.190:35913)
0925 11:00:04.123044 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.123051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.136605 (+ 13554us) server_negotiation.cc:407] Connection header received
0925 11:00:04.136869 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.136876 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.136972 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.137097 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:04.144830 (+ 7733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34572: BlockingRecv error: recv got EOF from 127.0.0.1:34572 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:00:04.197623 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:04.178568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35570 (local address 127.31.62.189:44389)
0925 11:00:04.178786 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.178792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.189523 (+ 10731us) server_negotiation.cc:407] Connection header received
0925 11:00:04.189767 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.189774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.189991 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.190167 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:04.190924 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.192088 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.194189 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.194890 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.194990 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:04.195773 (+ 783us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:04.196499 (+ 726us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:04.196624 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:04.197179 (+ 555us) server_negotiation.cc:299] Negotiation successful
0925 11:00:04.197274 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260925 11:00:04.247346 22181 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:04.239051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50126 (local address 127.31.62.188:42849)
0925 11:00:04.239255 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.239261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.245583 (+ 6322us) server_negotiation.cc:407] Connection header received
0925 11:00:04.245835 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.245842 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.245993 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.246131 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:04.247089 (+ 958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50126: BlockingRecv error: recv got EOF from 127.0.0.1:50126 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 11:00:04.264030 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:04.251915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35584 (local address 127.31.62.189:44389)
0925 11:00:04.252127 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.252133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.257900 (+ 5767us) server_negotiation.cc:407] Connection header received
0925 11:00:04.258130 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.258137 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.258234 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.258342 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:04.258817 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.259955 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.261913 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.262630 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.262776 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:04.263242 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:04.263404 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:04.263492 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:04.263628 (+ 136us) server_negotiation.cc:299] Negotiation successful
0925 11:00:04.263719 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260925 11:00:04.274807 22181 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:04.272641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50128 (local address 127.31.62.188:42849)
0925 11:00:04.272869 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.272876 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.273355 (+ 479us) server_negotiation.cc:407] Connection header received
0925 11:00:04.273565 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.273572 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.273673 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.273781 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:04.274573 (+ 792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50128: BlockingRecv error: recv got EOF from 127.0.0.1:50128 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:00:04.287016 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:04.279802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35596 (local address 127.31.62.189:44389)
0925 11:00:04.280000 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.280006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.280549 (+ 543us) server_negotiation.cc:407] Connection header received
0925 11:00:04.280748 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.280754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.280894 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.280989 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:04.281603 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.282860 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.284820 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.285543 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.285686 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:04.286171 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:04.286335 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:04.286421 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:04.286571 (+ 150us) server_negotiation.cc:299] Negotiation successful
0925 11:00:04.286668 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260925 11:00:04.297148 22181 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:04.294931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50130 (local address 127.31.62.188:42849)
0925 11:00:04.295110 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.295117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.295619 (+ 502us) server_negotiation.cc:407] Connection header received
0925 11:00:04.295821 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.295828 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.295931 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.296041 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:04.296904 (+ 863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50130: BlockingRecv error: recv got EOF from 127.0.0.1:50130 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 11:00:04.312633 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:04.305546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35612 (local address 127.31.62.189:44389)
0925 11:00:04.305796 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.305802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.306318 (+ 516us) server_negotiation.cc:407] Connection header received
0925 11:00:04.306533 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.306539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.306633 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.306735 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:04.307231 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.308439 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.310527 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.311216 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.311345 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:04.311839 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:04.311991 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:04.312089 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:04.312242 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 11:00:04.312333 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
W20260925 11:00:04.325521 22181 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:04.323104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50144 (local address 127.31.62.188:42849)
0925 11:00:04.323378 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.323385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.323947 (+ 562us) server_negotiation.cc:407] Connection header received
0925 11:00:04.324140 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.324147 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.324248 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.324364 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:04.325275 (+ 911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50144: BlockingRecv error: recv got EOF from 127.0.0.1:50144 (error 108)
Metrics: {"server-negotiator.queue_time_us":148}
I20260925 11:00:04.348500 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:04.339453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35616 (local address 127.31.62.189:44389)
0925 11:00:04.339641 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.339650 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.340554 (+ 904us) server_negotiation.cc:407] Connection header received
0925 11:00:04.340913 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.340920 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.341009 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.341080 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:04.342669 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.344055 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.346155 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.346826 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.346964 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:04.347570 (+ 606us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:04.347743 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:04.347827 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:04.348077 (+ 250us) server_negotiation.cc:299] Negotiation successful
0925 11:00:04.348165 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260925 11:00:04.367337 22181 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:04.365048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50154 (local address 127.31.62.188:42849)
0925 11:00:04.365223 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.365229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.365814 (+ 585us) server_negotiation.cc:407] Connection header received
0925 11:00:04.365983 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.365990 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.366100 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.366253 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:04.367076 (+ 823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50154: BlockingRecv error: recv got EOF from 127.0.0.1:50154 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:00:04.397996 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:04.390525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35628 (local address 127.31.62.189:44389)
0925 11:00:04.390712 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.390718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.391375 (+ 657us) server_negotiation.cc:407] Connection header received
0925 11:00:04.391607 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.391614 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.391723 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.391856 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:04.392378 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.393715 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.395709 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.396433 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.396601 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:04.397193 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:04.397347 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:04.397464 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:04.397604 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 11:00:04.397695 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260925 11:00:04.409613 22181 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:04.406899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50164 (local address 127.31.62.188:42849)
0925 11:00:04.407091 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.407098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.407688 (+ 590us) server_negotiation.cc:407] Connection header received
0925 11:00:04.408060 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.408067 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.408165 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.408244 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:04.409340 (+ 1096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50164: BlockingRecv error: recv got EOF from 127.0.0.1:50164 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 11:00:04.452880 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:04.445864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35634 (local address 127.31.62.189:44389)
0925 11:00:04.446041 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.446047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.446567 (+ 520us) server_negotiation.cc:407] Connection header received
0925 11:00:04.446838 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.446845 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.446939 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.447054 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:04.447485 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.448636 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.450688 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.451407 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.451570 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:04.451993 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:04.452144 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:04.452248 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:04.452393 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 11:00:04.452484 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260925 11:00:04.460654 22181 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:04.458801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50178 (local address 127.31.62.188:42849)
0925 11:00:04.458997 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.459003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.459302 (+ 299us) server_negotiation.cc:407] Connection header received
0925 11:00:04.459446 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.459452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.459563 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.459682 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:04.460413 (+ 731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50178: BlockingRecv error: recv got EOF from 127.0.0.1:50178 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 11:00:04.533905 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:04.528012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35644 (local address 127.31.62.189:44389)
0925 11:00:04.528622 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.528632 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.528654 (+ 22us) server_negotiation.cc:407] Connection header received
0925 11:00:04.528908 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.528914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.528984 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.529064 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:04.529655 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.530495 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.532140 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.532589 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.532699 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:04.533181 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:04.533306 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:04.533379 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:04.533520 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 11:00:04.533616 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469}
W20260925 11:00:04.543503 22181 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:04.541661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50182 (local address 127.31.62.188:42849)
0925 11:00:04.541859 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.541865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.542249 (+ 384us) server_negotiation.cc:407] Connection header received
0925 11:00:04.542375 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.542381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.542456 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.542544 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:04.543329 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50182: BlockingRecv error: recv got EOF from 127.0.0.1:50182 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 11:00:04.639158 22195 catalog_manager.cc:1259] Loaded cluster ID: fdbaeba6a2b3470192db4aa1f5b443e5
I20260925 11:00:04.639257 22195 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b: loading cluster ID for follower catalog manager: success
I20260925 11:00:04.641853 22195 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b: acquiring CA information for follower catalog manager: success
I20260925 11:00:04.643637 22195 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5dd0424f0b5f48bba832b7423523222b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 11:00:04.680922 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:04.675272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35652 (local address 127.31.62.189:44389)
0925 11:00:04.675504 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.675510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.675823 (+ 313us) server_negotiation.cc:407] Connection header received
0925 11:00:04.676082 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.676086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.676165 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.676255 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:04.676750 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.677567 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.679083 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.679506 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.679613 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:04.680122 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:04.680258 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:04.680356 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:04.680528 (+ 172us) server_negotiation.cc:299] Negotiation successful
0925 11:00:04.680600 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260925 11:00:04.692214 22181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:04.686612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50188 (local address 127.31.62.188:42849)
0925 11:00:04.686816 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.686823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.687231 (+ 408us) server_negotiation.cc:407] Connection header received
0925 11:00:04.687384 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.687389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.687538 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.687636 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:04.688057 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.688753 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.690062 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.690487 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.690579 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:04.691087 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:04.691639 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:04.691752 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:04.691832 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 11:00:04.691947 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:00:04.705706 22181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:04.698215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50198 (local address 127.31.62.188:42849)
0925 11:00:04.698413 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.698420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.700098 (+ 1678us) server_negotiation.cc:407] Connection header received
0925 11:00:04.700183 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.700189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.700272 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.700376 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:04.701108 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.702041 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.703736 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.704332 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.704461 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:04.704997 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:04.705120 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:04.705203 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:04.705333 (+ 130us) server_negotiation.cc:299] Negotiation successful
0925 11:00:04.705412 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 11:00:04.707036 22351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:04.698031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34576 (local address 127.31.62.190:35913)
0925 11:00:04.700408 (+ 2377us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.700414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.700431 (+ 17us) server_negotiation.cc:407] Connection header received
0925 11:00:04.700556 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.700561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.700776 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.700983 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:04.701630 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.702474 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.704353 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.704974 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.705071 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:04.705659 (+ 588us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:04.706098 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:04.706203 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:04.706654 (+ 451us) server_negotiation.cc:299] Negotiation successful
0925 11:00:04.706740 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2232,"thread_start_us":2144,"threads_started":1}
I20260925 11:00:04.707563 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:04.699184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35666 (local address 127.31.62.189:44389)
0925 11:00:04.699359 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:04.699365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:04.701632 (+ 2267us) server_negotiation.cc:407] Connection header received
0925 11:00:04.701897 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:04.701903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:04.702002 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:04.702091 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:04.702715 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.703787 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.705721 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:04.706341 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:04.706451 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:04.706874 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:04.706991 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:04.707077 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:04.707233 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 11:00:04.707315 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:00:04.712059 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 21963
I20260925 11:00:04.712992 22025 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:04.821995 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 21963
I20260925 11:00:04.838761 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 22037
I20260925 11:00:04.839821 22099 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:04.945742 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 22037
I20260925 11:00:04.962131 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 22111
I20260925 11:00:04.963245 22173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:05.070803 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 22111
2026-09-25T11:00:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4267 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:00:05 dist-test-slave-nx7s krb5kdc[22359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:00:05 dist-test-slave-nx7s krb5kdc[22359](info): set up 2 sockets
Sep 25 11:00:05 dist-test-slave-nx7s krb5kdc[22359](info): commencing operation
krb5kdc: starting...
W20260925 11:00:06.101624 31994 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/test-user.keytab.
Sep 25 11:00:06 dist-test-slave-nx7s krb5kdc[22359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:00:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:00:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
Entry for principal kudu/127.31.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab.
I20260925 11:00:06.364338 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:44717
--webserver_interface=127.31.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:44717,127.31.62.189:38295,127.31.62.188:37079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:06.701713 22372 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 11:00:06.702055 22372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:06.702126 22372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:06.702206 22372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:06.709599 22372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:06.709702 22372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:06.709755 22372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:06.709805 22372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:00:06.709870 22372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:06.719910 22372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:44717,127.31.62.189:38295,127.31.62.188:37079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.190.keytab
--principal=kudu/127.31.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.190:44717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:06.721207 22372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:06.722525 22372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:06.734019 22377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:06.734097 22380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:06.734057 22378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:06.734934 22372 server_base.cc:1034] running on GCE node
I20260925 11:00:06.735708 22372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:06.737236 22372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:06.738484 22372 hybrid_clock.cc:648] HybridClock initialized: now 1790334006738412 us; error 34 us; skew 500 ppm
I20260925 11:00:06.738903 22372 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:06 dist-test-slave-nx7s krb5kdc[22359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334006, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:06.742175 22372 init.cc:377] Logged in from keytab as kudu/127.31.62.190@KRBTEST.COM (short username kudu)
I20260925 11:00:06.743700 22372 webserver.cc:460] Webserver started at http://127.31.62.190:42759/ using document root <none> and password file <none>
I20260925 11:00:06.744400 22372 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:06.744518 22372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:06.744835 22372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:06.747926 22372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/data/instance:
uuid: "5fa29530cc43477cbd4e3d168f55452e"
format_stamp: "Formatted at 2026-09-25 11:00:06 on dist-test-slave-nx7s"
server_key: "9a800427a195eddf9543e0d458c381a8"
server_key_iv: "cf3e43022fa8955f282beed77648bb4d"
server_key_version: "encryptionkey@0"
I20260925 11:00:06.748718 22372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/wal/instance:
uuid: "5fa29530cc43477cbd4e3d168f55452e"
format_stamp: "Formatted at 2026-09-25 11:00:06 on dist-test-slave-nx7s"
server_key: "9a800427a195eddf9543e0d458c381a8"
server_key_iv: "cf3e43022fa8955f282beed77648bb4d"
server_key_version: "encryptionkey@0"
I20260925 11:00:06.754171 22372 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260925 11:00:06.757800 22387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:06.759227 22372 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:00:06.759393 22372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/wal
uuid: "5fa29530cc43477cbd4e3d168f55452e"
format_stamp: "Formatted at 2026-09-25 11:00:06 on dist-test-slave-nx7s"
server_key: "9a800427a195eddf9543e0d458c381a8"
server_key_iv: "cf3e43022fa8955f282beed77648bb4d"
server_key_version: "encryptionkey@0"
I20260925 11:00:06.759610 22372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:06.777916 22372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:06.782315 22372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:06.782579 22372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:06.800840 22372 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.190:44717
I20260925 11:00:06.800889 22438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.190:44717 every 8 connection(s)
I20260925 11:00:06.802286 22372 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/data/info.pb
I20260925 11:00:06.802654 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 22372
I20260925 11:00:06.802819 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-0/wal/instance
I20260925 11:00:06.803304 31994 external_mini_cluster.cc:1428] Setting key b0aa2e0d8bbfc7f5bf69cafe72e9ab82
I20260925 11:00:06.806427 22439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:06.813840 22439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.62.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.189@KRBTEST.COM" created.
Sep 25 11:00:06 dist-test-slave-nx7s krb5kdc[22359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334006, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 11:00:06.836575 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:06.817262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44717 (local address 127.0.0.1:40954)
0925 11:00:06.818130 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:06.818184 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:06.820424 (+ 2240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:06.821354 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:06.821361 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:06.822070 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:06.822422 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:06.822441 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:06.823989 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:06.823993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:06.825180 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:06.825192 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:06.826152 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:06.826160 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:06.826334 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:06.826947 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:06.826998 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:06.829488 (+ 2490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:06.832615 (+ 3127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:06.832627 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:06.832640 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:06.833090 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:06.833475 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:06.833480 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:06.833484 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:06.833635 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:06.834427 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:06.834440 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:06.834668 (+ 228us) client_negotiation.cc:769] Sending connection context
0925 11:00:06.834902 (+ 234us) client_negotiation.cc:240] Negotiation successful
0925 11:00:06.835573 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":120,"thread_start_us":351,"threads_started":1}
I20260925 11:00:06.836613 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:06.817262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40954 (local address 127.31.62.190:44717)
0925 11:00:06.819125 (+ 1863us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:06.819134 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:06.819191 (+ 57us) server_negotiation.cc:407] Connection header received
0925 11:00:06.820816 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:06.820822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:06.821131 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:06.821314 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:06.822618 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:06.823840 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:06.825308 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:06.825919 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:06.826104 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:06.829717 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:06.829725 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:06.829738 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:06.829846 (+ 108us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:06.832493 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:06.833216 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:06.833222 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:06.833224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:06.833374 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:06.833739 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:06.833744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:06.833746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:06.834305 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:06.834525 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:06.835090 (+ 565us) server_negotiation.cc:299] Negotiation successful
0925 11:00:06.835360 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1624,"thread_start_us":1292,"threads_started":1}
I20260925 11:00:06.839954 22439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:00:06.841159 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:06.840728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:38295 (local address 127.0.0.1:47274)
0925 11:00:06.840922 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:06.841033 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260925 11:00:06.841465 22390 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
I20260925 11:00:06.843084 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:06.842693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:38295 (local address 127.0.0.1:47286)
0925 11:00:06.842855 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:06.842931 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260925 11:00:06.843277 22439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.189:38295: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.189.keytab.
Entry for principal kudu/127.31.62.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.189.keytab.
I20260925 11:00:06.861131 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.189:38295
--webserver_interface=127.31.62.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.189.keytab
--principal=kudu/127.31.62.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:44717,127.31.62.189:38295,127.31.62.188:37079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:00:06.892656 22439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } attempt: 1
I20260925 11:00:06.893998 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:06.893478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:38295 (local address 127.0.0.1:47300)
0925 11:00:06.893686 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:06.893860 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260925 11:00:06.895135 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:06.894647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:38295 (local address 127.0.0.1:47314)
0925 11:00:06.894807 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:06.894948 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260925 11:00:06.895323 22439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.189:38295: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
I20260925 11:00:06.963577 22439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } attempt: 2
I20260925 11:00:06.964953 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:06.964383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:38295 (local address 127.0.0.1:47320)
0925 11:00:06.964594 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:06.964692 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260925 11:00:06.966037 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:06.965618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:38295 (local address 127.0.0.1:47324)
0925 11:00:06.965814 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:06.965902 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 11:00:06.966240 22439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.189:38295: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
I20260925 11:00:07.057536 22439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } attempt: 3
I20260925 11:00:07.058707 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.058271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:38295 (local address 127.0.0.1:47330)
0925 11:00:07.058473 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.058569 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260925 11:00:07.059921 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.059511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:38295 (local address 127.0.0.1:47334)
0925 11:00:07.059688 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.059771 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260925 11:00:07.060189 22439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.189:38295: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
W20260925 11:00:07.197907 22446 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 11:00:07.198273 22446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:07.198383 22446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:07.198508 22446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 11:00:07.203444 22439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } attempt: 4
I20260925 11:00:07.204999 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.204267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:38295 (local address 127.0.0.1:47350)
0925 11:00:07.204639 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.204739 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216}
I20260925 11:00:07.206166 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.205689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:38295 (local address 127.0.0.1:47364)
0925 11:00:07.205923 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.206022 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260925 11:00:07.206400 22439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.189:38295: Network error: Client connection negotiation failed: client connection to 127.31.62.189:38295: connect: Connection refused (error 111)
W20260925 11:00:07.207196 22446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:07.207288 22446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:07.207360 22446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:07.207423 22446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:00:07.207506 22446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:07.218039 22446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:44717,127.31.62.189:38295,127.31.62.188:37079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.189.keytab
--principal=kudu/127.31.62.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.189:38295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.62.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:07.219421 22446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:07.220878 22446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:07.233346 22452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:07.233428 22454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:07.233389 22451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:07.234292 22446 server_base.cc:1034] running on GCE node
I20260925 11:00:07.235200 22446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:07.236747 22446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:07.238015 22446 hybrid_clock.cc:648] HybridClock initialized: now 1790334007237948 us; error 65 us; skew 500 ppm
I20260925 11:00:07.238406 22446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:07 dist-test-slave-nx7s krb5kdc[22359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334007, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:07.241947 22446 init.cc:377] Logged in from keytab as kudu/127.31.62.189@KRBTEST.COM (short username kudu)
I20260925 11:00:07.243562 22446 webserver.cc:460] Webserver started at http://127.31.62.189:42321/ using document root <none> and password file <none>
I20260925 11:00:07.244282 22446 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:07.244388 22446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:07.244716 22446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:07.247880 22446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/data/instance:
uuid: "f0977cdf145a4bdfb8cba7dbe1593c20"
format_stamp: "Formatted at 2026-09-25 11:00:07 on dist-test-slave-nx7s"
server_key: "0f99814ccb2840ed53c85bcddf0d7a63"
server_key_iv: "3c0798863f768cb794d83e7ebcd0f740"
server_key_version: "encryptionkey@0"
I20260925 11:00:07.248711 22446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/wal/instance:
uuid: "f0977cdf145a4bdfb8cba7dbe1593c20"
format_stamp: "Formatted at 2026-09-25 11:00:07 on dist-test-slave-nx7s"
server_key: "0f99814ccb2840ed53c85bcddf0d7a63"
server_key_iv: "3c0798863f768cb794d83e7ebcd0f740"
server_key_version: "encryptionkey@0"
I20260925 11:00:07.254221 22446 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 11:00:07.258312 22461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:07.259929 22446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260925 11:00:07.260135 22446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/wal
uuid: "f0977cdf145a4bdfb8cba7dbe1593c20"
format_stamp: "Formatted at 2026-09-25 11:00:07 on dist-test-slave-nx7s"
server_key: "0f99814ccb2840ed53c85bcddf0d7a63"
server_key_iv: "3c0798863f768cb794d83e7ebcd0f740"
server_key_version: "encryptionkey@0"
I20260925 11:00:07.260378 22446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:07.287827 22446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:07.292193 22446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:07.292464 22446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:07.310122 22446 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.189:38295
I20260925 11:00:07.310165 22513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.189:38295 every 8 connection(s)
I20260925 11:00:07.311655 22446 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/data/info.pb
I20260925 11:00:07.315588 22514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:07.315999 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 22446
I20260925 11:00:07.316185 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-1/wal/instance
I20260925 11:00:07.316710 31994 external_mini_cluster.cc:1428] Setting key 25b3ab66e1026ac779e271e7f5275049
I20260925 11:00:07.324728 22514 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:00:07 dist-test-slave-nx7s krb5kdc[22359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334007, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.189@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.62.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.62.188@KRBTEST.COM" created.
I20260925 11:00:07.346550 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.327930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40968 (local address 127.31.62.190:44717)
0925 11:00:07.328131 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:07.328136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:07.329849 (+ 1713us) server_negotiation.cc:407] Connection header received
0925 11:00:07.331799 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:07.331804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:07.331894 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:07.331980 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:07.333627 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.334313 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.335907 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.336320 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.336442 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.340381 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:07.340387 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:07.340390 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:07.340428 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:07.343119 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.344036 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.344040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.344042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.344187 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.344725 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.344731 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.344735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.345097 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:07.345219 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:07.346110 (+ 891us) server_negotiation.cc:299] Negotiation successful
0925 11:00:07.346298 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260925 11:00:07.347978 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.328756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44717 (local address 127.0.0.1:40968)
0925 11:00:07.329685 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.329724 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:07.331490 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:07.332245 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:07.332253 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:07.332795 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:07.333442 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:07.333464 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.334479 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.334483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:07.335748 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:07.335761 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.336571 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.336579 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:07.336807 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.337543 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:07.337571 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:07.340093 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:07.343266 (+ 3173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:07.343275 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:07.343295 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:07.343908 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:07.344328 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:07.344333 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:07.344336 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:07.344597 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:07.345238 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:07.345246 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:07.345723 (+ 477us) client_negotiation.cc:769] Sending connection context
0925 11:00:07.346035 (+ 312us) client_negotiation.cc:240] Negotiation successful
0925 11:00:07.346594 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":337,"threads_started":1}
I20260925 11:00:07.349790 22514 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:00:07 dist-test-slave-nx7s krb5kdc[22359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334007, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.189@KRBTEST.COM for kudu/127.31.62.189@KRBTEST.COM
I20260925 11:00:07.368597 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.350557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:38295 (local address 127.0.0.1:47368)
0925 11:00:07.350717 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.350735 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:07.350859 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:07.352475 (+ 1616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:07.352481 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:07.352988 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:07.353173 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:07.353185 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.355380 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.355386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:07.356714 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:07.356731 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.358128 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.358133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:07.358278 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.358892 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:07.358914 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:07.361683 (+ 2769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:07.365903 (+ 4220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:07.365908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:07.365912 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:07.366288 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:07.366691 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:07.366695 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:07.366699 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:07.366834 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:07.367666 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:07.367670 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:07.367815 (+ 145us) client_negotiation.cc:769] Sending connection context
0925 11:00:07.367949 (+ 134us) client_negotiation.cc:240] Negotiation successful
0925 11:00:07.368230 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 11:00:07.369660 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.350944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47368 (local address 127.31.62.189:38295)
0925 11:00:07.351661 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:07.351669 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:07.351699 (+ 30us) server_negotiation.cc:407] Connection header received
0925 11:00:07.351752 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:07.351760 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:07.352007 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:07.352154 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:07.353371 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.355186 (+ 1815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.356908 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.357776 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.357897 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.361934 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:07.361952 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:07.361974 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:07.362017 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:07.365774 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.366419 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.366429 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.366433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.366584 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.366938 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.366943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.366946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.367395 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:07.367617 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:07.369002 (+ 1385us) server_negotiation.cc:299] Negotiation successful
0925 11:00:07.369253 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":391,"threads_started":1}
I20260925 11:00:07.372537 22514 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:00:07.374178 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.373639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49224)
0925 11:00:07.373837 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.374001 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 11:00:07.374516 22465 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
I20260925 11:00:07.376063 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.375719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49232)
0925 11:00:07.375853 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.375938 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260925 11:00:07.376281 22514 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:37079: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.62.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.188.keytab.
Entry for principal kudu/127.31.62.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.188.keytab.
I20260925 11:00:07.383512 31994 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
/tmp/dist-test-task9q_Uwc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.188:37079
--webserver_interface=127.31.62.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.188.keytab
--principal=kudu/127.31.62.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.62.148:44185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.62.190:44717,127.31.62.189:38295,127.31.62.188:37079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:00:07.425602 22514 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } attempt: 1
I20260925 11:00:07.426741 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.426285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49242)
0925 11:00:07.426479 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.426588 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260925 11:00:07.428478 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.428122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49254)
0925 11:00:07.428262 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.428343 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 11:00:07.428700 22514 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:37079: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
I20260925 11:00:07.496959 22514 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } attempt: 2
I20260925 11:00:07.498165 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.497700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49266)
0925 11:00:07.497905 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.498006 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260925 11:00:07.499154 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.498806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49270)
0925 11:00:07.498958 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.499033 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260925 11:00:07.499368 22514 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:37079: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
I20260925 11:00:07.505662 22439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } attempt: 5
Sep 25 11:00:07 dist-test-slave-nx7s krb5kdc[22359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334006, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for kudu/127.31.62.189@KRBTEST.COM
I20260925 11:00:07.519443 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.506599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47372 (local address 127.31.62.189:38295)
0925 11:00:07.506804 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:07.506810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:07.506830 (+ 20us) server_negotiation.cc:407] Connection header received
0925 11:00:07.506978 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:07.506984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:07.507070 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:07.507158 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:07.507972 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.508677 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.509891 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.510332 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.510445 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.513931 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:07.513936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:07.513939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:07.513978 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:07.516726 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.517648 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.517652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.517654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.517787 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.518172 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.518175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.518177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.518455 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:07.518566 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:07.519031 (+ 465us) server_negotiation.cc:299] Negotiation successful
0925 11:00:07.519212 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260925 11:00:07.519701 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.506488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:38295 (local address 127.0.0.1:47372)
0925 11:00:07.506680 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.506712 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:07.506855 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:07.507218 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:07.507223 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:07.507627 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:07.507821 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:07.507831 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.508818 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.508822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:07.509739 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:07.509752 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.510490 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.510497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:07.510703 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.511446 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:07.511467 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:07.513676 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:07.516934 (+ 3258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:07.516940 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:07.516944 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:07.517518 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:07.517895 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:07.517900 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:07.517904 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:07.518078 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:07.518566 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:07.518571 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:07.518751 (+ 180us) client_negotiation.cc:769] Sending connection context
0925 11:00:07.518923 (+ 172us) client_negotiation.cc:240] Negotiation successful
0925 11:00:07.519221 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260925 11:00:07.520753 22439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:00:07.521991 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.521606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49272)
0925 11:00:07.521765 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.521843 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260925 11:00:07.522862 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.522584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49278)
0925 11:00:07.522693 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.522769 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 11:00:07.523051 22439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:37079: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
I20260925 11:00:07.574272 22439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } attempt: 1
I20260925 11:00:07.575335 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.574879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49288)
0925 11:00:07.575090 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.575176 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 11:00:07.576292 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.575980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49298)
0925 11:00:07.576118 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.576185 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260925 11:00:07.576507 22439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:37079: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
I20260925 11:00:07.590569 22514 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } attempt: 3
I20260925 11:00:07.591692 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.591243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49302)
0925 11:00:07.591445 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.591539 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260925 11:00:07.592715 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.592302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49314)
0925 11:00:07.592527 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.592600 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260925 11:00:07.592934 22514 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:37079: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
I20260925 11:00:07.644716 22439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } attempt: 2
I20260925 11:00:07.645938 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.645468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49324)
0925 11:00:07.645688 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.645778 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260925 11:00:07.646888 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.646584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49336)
0925 11:00:07.646701 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.646770 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 11:00:07.647198 22439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:37079: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
W20260925 11:00:07.718281 22521 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 11:00:07.718639 22521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:07.718732 22521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:07.718845 22521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:07.726265 22521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:07.726382 22521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:07.726462 22521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:07.726524 22521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:00:07.726615 22521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:07.736141 22514 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } attempt: 4
I20260925 11:00:07.736542 22521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.62.148:44185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.62.190:44717,127.31.62.189:38295,127.31.62.188:37079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/krb5kdc/kudu_127.31.62.188.keytab
--principal=kudu/127.31.62.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.62.188:37079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.62.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:07.737268 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.736785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49342)
0925 11:00:07.737037 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.737124 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260925 11:00:07.738153 22521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:07.738220 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.737907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49352)
0925 11:00:07.738044 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.738112 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260925 11:00:07.738438 22514 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:37079: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
I20260925 11:00:07.739481 22521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:07.750833 22526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:07.750859 22527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:07.750799 22529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:07.753392 22439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } attempt: 3
I20260925 11:00:07.754456 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.753961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49362)
0925 11:00:07.754173 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.754269 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260925 11:00:07.754714 22521 server_base.cc:1034] running on GCE node
I20260925 11:00:07.755539 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.755154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49368)
0925 11:00:07.755351 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.755430 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260925 11:00:07.755662 22521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260925 11:00:07.755741 22439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.62.188:37079: Network error: Client connection negotiation failed: client connection to 127.31.62.188:37079: connect: Connection refused (error 111)
I20260925 11:00:07.757336 22521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:07.758576 22521 hybrid_clock.cc:648] HybridClock initialized: now 1790334007758488 us; error 58 us; skew 500 ppm
I20260925 11:00:07.758885 22521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:07 dist-test-slave-nx7s krb5kdc[22359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334007, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:07.762099 22521 init.cc:377] Logged in from keytab as kudu/127.31.62.188@KRBTEST.COM (short username kudu)
I20260925 11:00:07.763710 22521 webserver.cc:460] Webserver started at http://127.31.62.188:37031/ using document root <none> and password file <none>
I20260925 11:00:07.764524 22521 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:07.764614 22521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:07.764972 22521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:07.768173 22521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/data/instance:
uuid: "68a28538b08241c989a8533535bee7b7"
format_stamp: "Formatted at 2026-09-25 11:00:07 on dist-test-slave-nx7s"
server_key: "2e402f2436e44191cfdf2f01fff0c630"
server_key_iv: "d1984e4e81cbd99e9684c16a22068d83"
server_key_version: "encryptionkey@0"
I20260925 11:00:07.769089 22521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/wal/instance:
uuid: "68a28538b08241c989a8533535bee7b7"
format_stamp: "Formatted at 2026-09-25 11:00:07 on dist-test-slave-nx7s"
server_key: "2e402f2436e44191cfdf2f01fff0c630"
server_key_iv: "d1984e4e81cbd99e9684c16a22068d83"
server_key_version: "encryptionkey@0"
I20260925 11:00:07.774412 22521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 11:00:07.778223 22536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:07.779670 22521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:00:07.779850 22521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/data,/tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/wal
uuid: "68a28538b08241c989a8533535bee7b7"
format_stamp: "Formatted at 2026-09-25 11:00:07 on dist-test-slave-nx7s"
server_key: "2e402f2436e44191cfdf2f01fff0c630"
server_key_iv: "d1984e4e81cbd99e9684c16a22068d83"
server_key_version: "encryptionkey@0"
I20260925 11:00:07.780078 22521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:07.800048 22521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:07.804629 22521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:07.804935 22521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:07.823462 22521 rpc_server.cc:307] RPC server started. Bound to: 127.31.62.188:37079
I20260925 11:00:07.823468 22587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.62.188:37079 every 8 connection(s)
I20260925 11:00:07.824966 22521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/data/info.pb
I20260925 11:00:07.829057 22588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:07.830339 31994 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu as pid 22521
I20260925 11:00:07.830530 31994 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9q_Uwc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790333647382755-31994-0/minicluster-data/master-2/wal/instance
I20260925 11:00:07.831010 31994 external_mini_cluster.cc:1428] Setting key 046a050e1cce6bbbe5f5052bd5daec1a
I20260925 11:00:07.839040 22588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:00:07 dist-test-slave-nx7s krb5kdc[22359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.188@KRBTEST.COM
Sep 25 11:00:07 dist-test-slave-nx7s krb5kdc[22359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334007, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.188@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 11:00:07.855429 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.833084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49380 (local address 127.31.62.188:37079)
0925 11:00:07.833951 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:07.833971 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:07.834030 (+ 59us) server_negotiation.cc:407] Connection header received
0925 11:00:07.835694 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:07.835728 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:07.836368 (+ 640us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:07.836920 (+ 552us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:07.838314 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.840333 (+ 2019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.841961 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.842853 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.843044 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.847554 (+ 4510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:07.847564 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:07.847577 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:07.847625 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:07.850824 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.851748 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.851755 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.851763 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.851911 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.852414 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.852418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.852421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.853071 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:07.853374 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:07.853862 (+ 488us) server_negotiation.cc:299] Negotiation successful
0925 11:00:07.854288 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":356,"threads_started":1}
I20260925 11:00:07.857038 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.842103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40978 (local address 127.31.62.190:44717)
0925 11:00:07.842354 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:07.842361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:07.843133 (+ 772us) server_negotiation.cc:407] Connection header received
0925 11:00:07.843192 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:07.843196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:07.843256 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:07.843326 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:07.844794 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.845527 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.847138 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.847596 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.847728 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.851487 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:07.851496 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:07.851503 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:07.851554 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:07.854365 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.855024 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.855030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.855032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.855164 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.855520 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.855523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.855525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.855829 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:07.855981 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:07.856417 (+ 436us) server_negotiation.cc:299] Negotiation successful
0925 11:00:07.856640 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 11:00:07.857080 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.842067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.190:44717 (local address 127.0.0.1:40978)
0925 11:00:07.842798 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.842847 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:07.843014 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:07.843417 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:07.843424 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:07.844284 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:07.844628 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:07.844647 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.845678 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.845684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:07.846968 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:07.846984 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.847809 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.847816 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:07.848046 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.848638 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:07.848658 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:07.851233 (+ 2575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:07.854489 (+ 3256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:07.854501 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:07.854509 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:07.854905 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:07.855263 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:07.855269 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:07.855273 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:07.855412 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:07.855957 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:07.855966 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:07.856140 (+ 174us) client_negotiation.cc:769] Sending connection context
0925 11:00:07.856299 (+ 159us) client_negotiation.cc:240] Negotiation successful
0925 11:00:07.856634 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":303,"threads_started":1}
I20260925 11:00:07.858613 22588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:00:07 dist-test-slave-nx7s krb5kdc[22359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334007, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.188@KRBTEST.COM for kudu/127.31.62.189@KRBTEST.COM
Sep 25 11:00:07 dist-test-slave-nx7s krb5kdc[22359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.190@KRBTEST.COM
I20260925 11:00:07.871996 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.859289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.189:38295 (local address 127.0.0.1:47380)
0925 11:00:07.859555 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.859573 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:07.859677 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:07.860002 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:07.860007 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:07.860405 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:07.860587 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:07.860599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.861689 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.861693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:07.862606 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:07.862619 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.863409 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.863415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:07.863618 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.864192 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:07.864209 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:07.866440 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:07.869621 (+ 3181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:07.869627 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:07.869632 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:07.870006 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:07.870375 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:07.870380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:07.870383 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:07.870551 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:07.871047 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:07.871052 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:07.871201 (+ 149us) client_negotiation.cc:769] Sending connection context
0925 11:00:07.871329 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 11:00:07.871634 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260925 11:00:07.872026 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.859355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47380 (local address 127.31.62.189:38295)
0925 11:00:07.859516 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:07.859523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:07.859635 (+ 112us) server_negotiation.cc:407] Connection header received
0925 11:00:07.859797 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:07.859803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:07.859874 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:07.859954 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:07.860746 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.861538 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.862759 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.863234 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.863416 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.866643 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:07.866650 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:07.866655 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:07.866696 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:07.869475 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.870124 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.870128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.870131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.870268 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.870653 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.870658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.870661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.870922 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:07.871067 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:07.871430 (+ 363us) server_negotiation.cc:299] Negotiation successful
0925 11:00:07.871634 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 11:00:07.873129 22588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:00:07.873222 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.858627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40980 (local address 127.31.62.190:44717)
0925 11:00:07.858961 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:07.858967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:07.858986 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:00:07.859095 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:07.859101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:07.859171 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:07.859270 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:07.860393 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.861326 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.862563 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.863077 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.863168 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.867628 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:07.867633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:07.867636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:07.867671 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:07.870288 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.870960 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.870964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.870966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.871100 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.871464 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.871467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.871470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.871730 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:07.871986 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:07.872666 (+ 680us) server_negotiation.cc:299] Negotiation successful
0925 11:00:07.872975 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Sep 25 11:00:07 dist-test-slave-nx7s krb5kdc[22359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334007, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.188@KRBTEST.COM for kudu/127.31.62.188@KRBTEST.COM
Sep 25 11:00:07 dist-test-slave-nx7s krb5kdc[22359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.62.189@KRBTEST.COM
I20260925 11:00:07.886154 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.873979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49388)
0925 11:00:07.874153 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.874168 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:07.874246 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:07.874554 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:07.874558 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:07.874995 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:07.875163 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:07.875173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.876098 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.876102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:07.877104 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:07.877115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.877775 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.877781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:07.877991 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.878530 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:07.878553 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:07.880692 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:07.883719 (+ 3027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:07.883725 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:07.883729 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:07.884086 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:07.884514 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:07.884518 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:07.884520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:07.884644 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:07.885321 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:07.885327 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:07.885504 (+ 177us) client_negotiation.cc:769] Sending connection context
0925 11:00:07.885626 (+ 122us) client_negotiation.cc:240] Negotiation successful
0925 11:00:07.885866 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260925 11:00:07.886405 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.874195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49388 (local address 127.31.62.188:37079)
0925 11:00:07.874339 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:07.874344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:07.874356 (+ 12us) server_negotiation.cc:407] Connection header received
0925 11:00:07.874386 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:07.874389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:07.874440 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:07.874529 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:07.875288 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.875971 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.877228 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.877621 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.877709 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.880874 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:07.880880 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:07.880885 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:07.880917 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:07.883608 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.884202 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.884207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.884211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.884393 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.884728 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.884733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.884736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.885208 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:07.885330 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:07.885750 (+ 420us) server_negotiation.cc:299] Negotiation successful
0925 11:00:07.886032 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":47}
I20260925 11:00:07.889531 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.874770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47396 (local address 127.31.62.189:38295)
0925 11:00:07.874909 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:07.874912 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:07.874973 (+ 61us) server_negotiation.cc:407] Connection header received
0925 11:00:07.875107 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:07.875110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:07.875159 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:07.875232 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:07.876398 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.877109 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.878567 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.878983 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.879071 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.883486 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:07.883493 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:07.883498 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:07.883537 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:07.886770 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.887520 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.887524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.887527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.887690 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.888064 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.888067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.888070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.888408 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:07.888671 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:07.888989 (+ 318us) server_negotiation.cc:299] Negotiation successful
0925 11:00:07.889216 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 11:00:07.892539 22588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7: Bootstrap starting.
I20260925 11:00:07.894248 31994 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:00:07.896430 22588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:07.897789 22588 log.cc:824] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:07.897253 22414 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5fa29530cc43477cbd4e3d168f55452e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40980
I20260925 11:00:07.901089 22588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7: No bootstrap required, opened a new log
I20260925 11:00:07.911790 22588 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } }
I20260925 11:00:07.912256 22588 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68a28538b08241c989a8533535bee7b7, State: Initialized, Role: FOLLOWER
I20260925 11:00:07.912781 22588 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } }
I20260925 11:00:07.913805 22594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } } }
I20260925 11:00:07.914119 22594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:00:07.915125 22588 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7 [sys.catalog]: configured and running, proceeding with master startup.
W20260925 11:00:07.924369 22605 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 11:00:07.924525 22605 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 11:00:07.932921 22439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } attempt: 4
I20260925 11:00:07.922307 22521 master_runner.cc:186] Error getting master registration for 127.31.62.190:44717: OK, instance_id { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" instance_seqno: 1790334006795666 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 11:00:07.933336 22521 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 25 11:00:07 dist-test-slave-nx7s krb5kdc[22359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334006, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.190@KRBTEST.COM for kudu/127.31.62.188@KRBTEST.COM
I20260925 11:00:07.948227 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.933556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49400)
0925 11:00:07.933743 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:07.933763 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:07.933892 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:07.934649 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:07.934655 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:07.935054 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:07.935218 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:07.935228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.936769 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.936775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:07.937762 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:07.937776 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.938618 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.938625 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:07.938932 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.939504 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:07.939522 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:07.942110 (+ 2588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:07.945474 (+ 3364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:07.945480 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:07.945485 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:07.946008 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:07.946415 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:07.946419 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:07.946423 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:07.946573 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:07.947223 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:07.947227 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:07.947389 (+ 162us) client_negotiation.cc:769] Sending connection context
0925 11:00:07.947526 (+ 137us) client_negotiation.cc:240] Negotiation successful
0925 11:00:07.947814 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 11:00:07.948271 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:07.934129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49400 (local address 127.31.62.188:37079)
0925 11:00:07.934332 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:07.934338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:07.934359 (+ 21us) server_negotiation.cc:407] Connection header received
0925 11:00:07.934415 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:07.934420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:07.934501 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:07.934604 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:07.935373 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.936618 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.937903 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:07.938390 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:07.938594 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:07.942329 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:07.942335 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:07.942338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:07.942372 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:07.945294 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.946138 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.946144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.946148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.946302 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:07.946679 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:07.946684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:07.946688 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:07.947106 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:07.947271 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:07.947673 (+ 402us) server_negotiation.cc:299] Negotiation successful
0925 11:00:07.947923 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:00:07.954367 22439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e: Bootstrap starting.
I20260925 11:00:07.958504 22439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:07.959841 22439 log.cc:824] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:07.962958 22439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e: No bootstrap required, opened a new log
I20260925 11:00:07.970666 22439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } }
I20260925 11:00:07.971151 22439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa29530cc43477cbd4e3d168f55452e, State: Initialized, Role: FOLLOWER
I20260925 11:00:07.971683 22439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } }
I20260925 11:00:07.972980 22608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } } }
I20260925 11:00:07.973623 22608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:00:07.974257 22439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:00:07.974365 22414 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 11:00:07.974537 22414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:07.978214 22414 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } }
I20260925 11:00:07.981686 22414 leader_election.cc:288] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [CANDIDATE]: Term 1 election: Requested vote from peers f0977cdf145a4bdfb8cba7dbe1593c20 (127.31.62.189:38295), 68a28538b08241c989a8533535bee7b7 (127.31.62.188:37079)
I20260925 11:00:07.982582 22563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fa29530cc43477cbd4e3d168f55452e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "68a28538b08241c989a8533535bee7b7"
I20260925 11:00:07.983062 22563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:07.986658 22563 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fa29530cc43477cbd4e3d168f55452e in term 1.
I20260925 11:00:07.987318 22391 leader_election.cc:302] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa29530cc43477cbd4e3d168f55452e, 68a28538b08241c989a8533535bee7b7; no voters:
I20260925 11:00:07.987749 22608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:07.994804 22608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [term 1 LEADER]: Becoming Leader. State: Replica: 5fa29530cc43477cbd4e3d168f55452e, State: Running, Role: LEADER
I20260925 11:00:07.995603 22608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } }
I20260925 11:00:08.023557 22622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fa29530cc43477cbd4e3d168f55452e. Latest consensus state: current_term: 1 leader_uuid: "5fa29530cc43477cbd4e3d168f55452e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } } }
I20260925 11:00:08.024039 22622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:07.989140 22372 master_runner.cc:186] Error getting master registration for 127.31.62.189:38295: OK, instance_id { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" instance_seqno: 1790334007304956 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 11:00:08.037626 22514 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } attempt: 5
W20260925 11:00:08.035570 22393 tablet.cc:2369] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:00:08.038725 22619 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 11:00:08.038861 22619 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 11:00:08.039045 22626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:00:08.039182 22372 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 11:00:08.042985 22626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 11:00:08 dist-test-slave-nx7s krb5kdc[22359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334007, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.62.189@KRBTEST.COM for kudu/127.31.62.188@KRBTEST.COM
W20260925 11:00:08.057245 22542 tablet.cc:2369] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:00:08.062673 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.038358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.62.188:37079 (local address 127.0.0.1:49402)
0925 11:00:08.038937 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:08.038958 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:08.039097 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:08.040323 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:08.040327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:08.040882 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:08.041105 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:08.041119 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.042919 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.042925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:08.044282 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:08.044310 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.045641 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.045647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:08.045919 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.046916 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:08.046939 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:08.051396 (+ 4457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:08.056708 (+ 5312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:08.056714 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:08.056717 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:08.057341 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:08.060065 (+ 2724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:08.060070 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:08.060074 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:08.060265 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:08.061541 (+ 1276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:08.061547 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:08.061737 (+ 190us) client_negotiation.cc:769] Sending connection context
0925 11:00:08.061885 (+ 148us) client_negotiation.cc:240] Negotiation successful
0925 11:00:08.062238 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434}
I20260925 11:00:08.066316 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.039364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49402 (local address 127.31.62.188:37079)
0925 11:00:08.039552 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.039557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.039580 (+ 23us) server_negotiation.cc:407] Connection header received
0925 11:00:08.039649 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.039654 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.039743 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.039855 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:08.041619 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.042728 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.044582 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.045287 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.045420 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.052471 (+ 7051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:08.052479 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:08.052485 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:08.052532 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:08.056449 (+ 3917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:08.059549 (+ 3100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:08.059555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:08.059558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:08.059748 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:08.060646 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:08.060651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:08.060655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:08.061064 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:08.061221 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.064210 (+ 2989us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.065408 (+ 1198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 11:00:08.073024 22514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20: Bootstrap starting.
I20260925 11:00:08.079754 22514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:08.081261 22514 log.cc:824] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:08.098067 22514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20: No bootstrap required, opened a new log
I20260925 11:00:08.123523 22563 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7 [term 1 FOLLOWER]: Refusing update from remote peer 5fa29530cc43477cbd4e3d168f55452e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:08.124760 22608 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:08.133353 22514 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } }
I20260925 11:00:08.134130 22514 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0977cdf145a4bdfb8cba7dbe1593c20, State: Initialized, Role: FOLLOWER
I20260925 11:00:08.135047 22514 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } }
I20260925 11:00:08.152073 22635 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } } }
I20260925 11:00:08.153803 22635 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:00:08.154511 22489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fa29530cc43477cbd4e3d168f55452e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20"
I20260925 11:00:08.154933 22488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:08.154958 22514 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:00:08.159008 22488 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20 [term 1 FOLLOWER]: Refusing update from remote peer 5fa29530cc43477cbd4e3d168f55452e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:08.160238 22608 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:08.163098 22489 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fa29530cc43477cbd4e3d168f55452e in term 1.
I20260925 11:00:08.165321 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.041362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.31.62.188:37079)
0925 11:00:08.077880 (+ 36518us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.077887 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.090563 (+ 12676us) server_negotiation.cc:407] Connection header received
0925 11:00:08.090651 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.090658 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.090776 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.090878 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:08.103771 (+ 12893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.104984 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.117313 (+ 12329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.118094 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.118273 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.141036 (+ 22763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:08.141046 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:08.141051 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:08.141105 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:08.146606 (+ 5501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:08.148224 (+ 1618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:08.148257 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:08.148261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:08.148550 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:08.149440 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:08.149446 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:08.149450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:08.150130 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:08.150617 (+ 487us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.153746 (+ 3129us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.154217 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36381,"thread_start_us":62177,"threads_started":1}
I20260925 11:00:08.166635 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.058202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.31.62.190:44717)
0925 11:00:08.058407 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.058413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.091567 (+ 33154us) server_negotiation.cc:407] Connection header received
0925 11:00:08.091841 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.091847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.091934 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.092005 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:08.105263 (+ 13258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.106416 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.115130 (+ 8714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.115923 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.116031 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.129349 (+ 13318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:08.129357 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:08.129362 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:08.129425 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:08.133466 (+ 4041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:08.136398 (+ 2932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:08.136405 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:08.136409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:08.136763 (+ 354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:08.151299 (+ 14536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:08.151307 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:08.151311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:08.151737 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:08.151979 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.165977 (+ 13998us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.166267 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 11:00:08.178398 22594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fa29530cc43477cbd4e3d168f55452e. Latest consensus state: current_term: 1 leader_uuid: "5fa29530cc43477cbd4e3d168f55452e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } } }
I20260925 11:00:08.178727 22594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:00:08.186332 22634 mvcc.cc:205] Tried to move back new op lower bound from 7333208097232961536 to 7333208096757260288. Current Snapshot: MvccSnapshot[applied={T|T < 7333208097232961536}]
I20260925 11:00:08.186734 22608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5fa29530cc43477cbd4e3d168f55452e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } } }
I20260925 11:00:08.187117 22608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:08.187579 22594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5fa29530cc43477cbd4e3d168f55452e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } } }
I20260925 11:00:08.187822 22594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:00:08.188584 22635 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fa29530cc43477cbd4e3d168f55452e. Latest consensus state: current_term: 1 leader_uuid: "5fa29530cc43477cbd4e3d168f55452e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } } }
I20260925 11:00:08.188891 22635 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:00:08.191032 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.078949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47404 (local address 127.31.62.189:38295)
0925 11:00:08.079120 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.079126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.110836 (+ 31710us) server_negotiation.cc:407] Connection header received
0925 11:00:08.115106 (+ 4270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.115113 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.115211 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.115339 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:08.124025 (+ 8686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.125173 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.137082 (+ 11909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.137842 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.137958 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.183425 (+ 45467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:08.183433 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:08.183437 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:08.183486 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:08.187446 (+ 3960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:08.188449 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:08.188455 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:08.188458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:08.188640 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:08.189157 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:08.189163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:08.189166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:08.189576 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:08.189821 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.190425 (+ 604us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.190675 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:00:08.191315 22622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5fa29530cc43477cbd4e3d168f55452e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } } }
I20260925 11:00:08.191571 22622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:08.194105 22635 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5fa29530cc43477cbd4e3d168f55452e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fa29530cc43477cbd4e3d168f55452e" member_type: VOTER last_known_addr { host: "127.31.62.190" port: 44717 } } peers { permanent_uuid: "f0977cdf145a4bdfb8cba7dbe1593c20" member_type: VOTER last_known_addr { host: "127.31.62.189" port: 38295 } } peers { permanent_uuid: "68a28538b08241c989a8533535bee7b7" member_type: VOTER last_known_addr { host: "127.31.62.188" port: 37079 } } }
I20260925 11:00:08.194381 22635 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:00:08.194593 22626 catalog_manager.cc:1347] Generated new cluster ID: 9d60a65aacd547519a058982580d59bf
I20260925 11:00:08.194684 22626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:00:08.212311 22626 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:00:08.214187 22626 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:00:08.272006 22626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fa29530cc43477cbd4e3d168f55452e: Generated new TSK 0
I20260925 11:00:08.273455 22626 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:00:08.294096 22652 catalog_manager.cc:1259] Loaded cluster ID: 9d60a65aacd547519a058982580d59bf
I20260925 11:00:08.294236 22652 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20: loading cluster ID for follower catalog manager: success
I20260925 11:00:08.297582 22652 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20: acquiring CA information for follower catalog manager: success
I20260925 11:00:08.299645 22652 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f0977cdf145a4bdfb8cba7dbe1593c20: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 11:00:08.409293 22443 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:08.383842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40996 (local address 127.31.62.190:44717)
0925 11:00:08.384066 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.384072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.404814 (+ 20742us) server_negotiation.cc:407] Connection header received
0925 11:00:08.407612 (+ 2798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.407620 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.407747 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.407839 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:08.409020 (+ 1181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40996: BlockingRecv error: recv got EOF from 127.0.0.1:40996 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260925 11:00:08.411603 22518 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:08.390176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47406 (local address 127.31.62.189:38295)
0925 11:00:08.390403 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.390409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.407170 (+ 16761us) server_negotiation.cc:407] Connection header received
0925 11:00:08.410056 (+ 2886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.410063 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.410170 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.410253 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:08.411343 (+ 1090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47406: BlockingRecv error: recv got EOF from 127.0.0.1:47406 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260925 11:00:08.414582 22591 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:08.402285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49434 (local address 127.31.62.188:37079)
0925 11:00:08.402488 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.402494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.412422 (+ 9928us) server_negotiation.cc:407] Connection header received
0925 11:00:08.413187 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.413193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.413291 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.413364 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:08.414328 (+ 964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49434: BlockingRecv error: recv got EOF from 127.0.0.1:49434 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 11:00:08.457913 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.441810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47410 (local address 127.31.62.189:38295)
0925 11:00:08.442015 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.442022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.449480 (+ 7458us) server_negotiation.cc:407] Connection header received
0925 11:00:08.450086 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.450093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.450343 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.450486 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:08.451260 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.452409 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.454639 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.455350 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.455460 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.456297 (+ 837us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:08.457094 (+ 797us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:08.457215 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.457463 (+ 248us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.457578 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260925 11:00:08.498507 22591 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:08.489100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49438 (local address 127.31.62.188:37079)
0925 11:00:08.489364 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.489371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.496315 (+ 6944us) server_negotiation.cc:407] Connection header received
0925 11:00:08.496864 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.496871 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.497011 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.497087 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:08.498232 (+ 1145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49438: BlockingRecv error: recv got EOF from 127.0.0.1:49438 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 11:00:08.510846 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.504122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47426 (local address 127.31.62.189:38295)
0925 11:00:08.504324 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.504330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.504666 (+ 336us) server_negotiation.cc:407] Connection header received
0925 11:00:08.504892 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.504898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.505005 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.505104 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:08.505589 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.506724 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.508694 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.509423 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.509563 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.510017 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:08.510200 (+ 183us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:08.510309 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.510438 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.510522 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260925 11:00:08.531268 22591 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:08.523492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49452 (local address 127.31.62.188:37079)
0925 11:00:08.523697 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.523704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.529636 (+ 5932us) server_negotiation.cc:407] Connection header received
0925 11:00:08.529865 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.529871 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.529980 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.530090 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:08.531019 (+ 929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49452: BlockingRecv error: recv got EOF from 127.0.0.1:49452 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 11:00:08.545926 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.538965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47442 (local address 127.31.62.189:38295)
0925 11:00:08.539150 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.539155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.539706 (+ 551us) server_negotiation.cc:407] Connection header received
0925 11:00:08.539892 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.539899 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.539992 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.540140 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:08.540623 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.541817 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.543732 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.544400 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.544554 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.545039 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:08.545196 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:08.545286 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.545493 (+ 207us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.545581 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260925 11:00:08.556007 22591 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:08.553686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49468 (local address 127.31.62.188:37079)
0925 11:00:08.553885 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.553892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.554467 (+ 575us) server_negotiation.cc:407] Connection header received
0925 11:00:08.554678 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.554684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.554810 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.554914 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:08.555750 (+ 836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49468: BlockingRecv error: recv got EOF from 127.0.0.1:49468 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 11:00:08.577998 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.570572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47458 (local address 127.31.62.189:38295)
0925 11:00:08.570776 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.570782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.571395 (+ 613us) server_negotiation.cc:407] Connection header received
0925 11:00:08.571602 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.571608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.571693 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.571785 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:08.572252 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.573361 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.575319 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.576052 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.576176 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.576730 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:08.576912 (+ 182us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:08.577008 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.577613 (+ 605us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.577708 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260925 11:00:08.586966 22591 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:08.584361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49470 (local address 127.31.62.188:37079)
0925 11:00:08.584665 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.584672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.585403 (+ 731us) server_negotiation.cc:407] Connection header received
0925 11:00:08.585633 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.585639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.585752 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.585867 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:08.586750 (+ 883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49470: BlockingRecv error: recv got EOF from 127.0.0.1:49470 (error 108)
Metrics: {"server-negotiator.queue_time_us":185}
I20260925 11:00:08.606113 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.598410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47472 (local address 127.31.62.189:38295)
0925 11:00:08.598622 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.598628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.599173 (+ 545us) server_negotiation.cc:407] Connection header received
0925 11:00:08.599715 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.599722 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.599821 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.599930 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:08.600403 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.601628 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.603720 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.604418 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.604577 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.605185 (+ 608us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:08.605366 (+ 181us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:08.605475 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.605674 (+ 199us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.605768 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260925 11:00:08.616878 22591 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:08.614482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49472 (local address 127.31.62.188:37079)
0925 11:00:08.614681 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.614687 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.615262 (+ 575us) server_negotiation.cc:407] Connection header received
0925 11:00:08.615440 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.615446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.615558 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.615701 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:08.616599 (+ 898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49472: BlockingRecv error: recv got EOF from 127.0.0.1:49472 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 11:00:08.649991 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.642815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47478 (local address 127.31.62.189:38295)
0925 11:00:08.643025 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.643031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.643601 (+ 570us) server_negotiation.cc:407] Connection header received
0925 11:00:08.643815 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.643822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.643923 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.644035 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:08.644534 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.645787 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.647764 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.648488 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.648657 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.649213 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:08.649356 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:08.649457 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.649613 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.649708 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260925 11:00:08.663074 22591 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:08.657522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49474 (local address 127.31.62.188:37079)
0925 11:00:08.657709 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.657715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.661438 (+ 3723us) server_negotiation.cc:407] Connection header received
0925 11:00:08.661647 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.661654 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.661769 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.661892 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:08.662833 (+ 941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49474: BlockingRecv error: recv got EOF from 127.0.0.1:49474 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 11:00:08.705497 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.699390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47490 (local address 127.31.62.189:38295)
0925 11:00:08.699623 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.699630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.699978 (+ 348us) server_negotiation.cc:407] Connection header received
0925 11:00:08.700146 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.700152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.700265 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.700406 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:08.700819 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.701970 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.703335 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.704057 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.704192 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.704637 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:08.704797 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:08.704926 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.705056 (+ 130us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.705148 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260925 11:00:08.713940 22591 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:08.712074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49478 (local address 127.31.62.188:37079)
0925 11:00:08.712272 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.712278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.712503 (+ 225us) server_negotiation.cc:407] Connection header received
0925 11:00:08.712696 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.712700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.712775 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.712881 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:08.713700 (+ 819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49478: BlockingRecv error: recv got EOF from 127.0.0.1:49478 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 11:00:08.786502 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.781284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47492 (local address 127.31.62.189:38295)
0925 11:00:08.781488 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.781494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.781757 (+ 263us) server_negotiation.cc:407] Connection header received
0925 11:00:08.782038 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.782042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.782111 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.782187 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:08.782572 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.783327 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.784708 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.785290 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.785410 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.785791 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:08.785914 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:08.785987 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.786130 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.786210 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260925 11:00:08.793560 22591 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:08.791637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49484 (local address 127.31.62.188:37079)
0925 11:00:08.791827 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.791833 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.792173 (+ 340us) server_negotiation.cc:407] Connection header received
0925 11:00:08.792401 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.792405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.792478 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.792561 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:08.793345 (+ 784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49484: BlockingRecv error: recv got EOF from 127.0.0.1:49484 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 11:00:08.926863 22605 catalog_manager.cc:1259] Loaded cluster ID: 9d60a65aacd547519a058982580d59bf
I20260925 11:00:08.926957 22605 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7: loading cluster ID for follower catalog manager: success
I20260925 11:00:08.929557 22605 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7: acquiring CA information for follower catalog manager: success
I20260925 11:00:08.931334 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.926263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47494 (local address 127.31.62.189:38295)
0925 11:00:08.926492 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.926498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.926839 (+ 341us) server_negotiation.cc:407] Connection header received
0925 11:00:08.927139 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.927144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.927215 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.927292 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:08.927736 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.928421 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.929797 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.930192 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.930309 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.930656 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:08.930771 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:08.930848 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.930980 (+ 132us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.931062 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 11:00:08.931437 22605 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 68a28538b08241c989a8533535bee7b7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 11:00:08.942816 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.937123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49494 (local address 127.31.62.188:37079)
0925 11:00:08.937329 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.937336 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.937674 (+ 338us) server_negotiation.cc:407] Connection header received
0925 11:00:08.937866 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.937890 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.938086 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.938198 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:08.938604 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.939286 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.940611 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.941077 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.941189 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.941695 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:08.942221 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:08.942355 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.942458 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.942543 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:00:08.961120 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.949296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47500 (local address 127.31.62.189:38295)
0925 11:00:08.949686 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.949692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.952951 (+ 3259us) server_negotiation.cc:407] Connection header received
0925 11:00:08.953497 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.953504 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.953605 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.953671 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:08.954520 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.955740 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.958966 (+ 3226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.959618 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.959774 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.960322 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:08.960429 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:08.960501 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.960729 (+ 228us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.960819 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261}
I20260925 11:00:08.965591 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.949083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49504 (local address 127.31.62.188:37079)
0925 11:00:08.949516 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.949524 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.959794 (+ 10270us) server_negotiation.cc:407] Connection header received
0925 11:00:08.960081 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.960087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.960174 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.960245 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:08.960991 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.961827 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.963528 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.964199 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.964321 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.964906 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:08.965009 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:08.965091 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.965234 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.965313 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260925 11:00:08.965647 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:08.949751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41010 (local address 127.31.62.190:44717)
0925 11:00:08.959341 (+ 9590us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:08.959347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:08.959366 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:00:08.959486 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:08.959492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:08.959681 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:08.959910 (+ 229us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:08.960544 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.961557 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.963373 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:08.963864 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:08.963969 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:08.964497 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:08.965074 (+ 577us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:08.965171 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:08.965271 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 11:00:08.965354 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9419,"thread_start_us":9296,"threads_started":1}
I20260925 11:00:08.976336 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 22372
I20260925 11:00:08.978057 22434 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:09.088258 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 22372
I20260925 11:00:09.105624 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 22446
I20260925 11:00:09.106671 22509 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:09.213860 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 22446
I20260925 11:00:09.231379 31994 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu pid 22521
I20260925 11:00:09.232621 22583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:09.339498 31994 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9q_Uwc/build/asan/bin/kudu with pid 22521
2026-09-25T11:00:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4269 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (8536 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (361918 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST